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Conciliis  et  discidiis  exercita  crebris© 

Coni  cere  ut  possis  ex  hocpprimordia  rerum 
Qual©  sit  in  magno  iactari  semper  inani© 

Bumtaxat  rerum  magnarum  parva  potest  res 
Exemplar©  dare  et  vestigia  notitiai© 

Hoc  etlam  magi s haec  animum  te  advertere  par  est 
Corpora^ qua©  in  soils  radiis  turbare  videntur 
Quod  tales  turba©  ruotus  quoque  materiai 
Significant  clandestinos  caecosque  subesse© 

Multa  videbis  enim  plagis  ibi  percita  caecis 
Commutax*©  viam  retroque  pulsa  revert! 

Nunc  hue*  nunc  illuc  in  cunetas  undique  partes0 

Titus  Lucre tiu3  Carus 

De  Rerum  NaturasVol ©IloVers  113^130* 


Let  us  observe  as  brightly  the  rays  of  the  sun 
Penetrate  in  streams  the  darkness  of  our  houses 
Thousands  of  tiny  bodies  dancing  in  space 

Approaching  each  other  and  parting  in  the  bright  light  of  the  sun© 
As  if  fighting  a battle  without  pause  through  the  ages e 
Like  an  army  of  soldiers  restlessly  warring * 

They  advance  and  retreat  in  motion  never  to  cease© 

May  you  conjecture  from  this  the  very  nature  of  matter^ 

How  it  is  ceaselessly  tossed  through  the  vastness  of  space© 

Thus  a phenomenons  small  as  it  seems  and  of  little  importance 
Often  does  indicate  things  highly  important  and  great© 

Hence  it  is  well  worthwhile  to  observe  these  bodies 
Whirling  and  dancing  without  rest  in  the  sunlight^ 

Sine©  such  irregular  motion  of  visible  bodies 

Is  a sure  indication  of  the  invisible  motion  of  matter© 

For  you  can  see  these  bodies  constantly  changing  direction^ 

Often  reversing  their  motion  all  of  a sudden 

And  propelled  by  invisible  impacts  moving  this  way  and  that  way© 
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FOREWORD 


Th©  theory  of  stochastic  processes  is  steadily  gaining 
in  importance  and  the  applications  ar©  ever  widening©  Never^ 
theless*  it  is  at  present  not  easy  to  study  this  subject#  since 
the  literature g although  extensive#  is  widely  scattered© 

This  situation  motivated  the  National  Bureau  of  Standards 
to  invite  Professor  Henry  B©  Mann  to  give  a series  of  lectures 
on  stochastic  processes  and  to  write  a monograph  on  the  subject© 
The  lectures  "were  given  in  the  period  from  March  1949  to  June 
1949 0 during  which  Dr©  Mann  was  a member  of  the  staff  of  th© 
Bureau® s Statistical  Engineering  Laboratory© 

It  is  well  known  that  the  theory  of  stochastic  processes 
depending  on  a continuous  parameter  can  be  developed  in  a 
satisfactory  way  by  studying  random  functions  or  by  consider^ 
ing  probability  measures  in  function  space©  The  author  of 
th©  present  monograph  has  however  adopted  a different  approach 
which  is  similar  to  the  definition  of  a stochastic  process 
given  by  E©  Slutsky p A random  variable  is  considered  to  be 
a symbol  with  which  a distribution  function  is  associated# 
and  a stochastic  process  is  then  defined  as  a set  of  random 
variables®  This  approach  loads  to  a t beery  which  for  many 
practical  purposes  is  equivalent  to  th©  direct  measure** 
theoretical  approach©  It  has  th©  advantage  that  the  tech® 
nicali ties  of  measure  theory  seem  loss  obstrusiv©  at  th© 
outset#  although  for  logical  completness  they  must  enter 

sooner  or  later  if  the  theory  is  to  be  developed  in  a w©ll=> 
rounded  way0 
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It  is  hoped  that  this  modest  little  volume,  written 
by  i distinguished  contemporary  mathematician,  will  b©  use- 
ful and  Interesting  in  various  vrays©  The  argument  is  addressed 
uncompromisingly  to  educated  mathematicians,  and  they  will 
not  fail  to  be  impressed  by  the  skillful  way  in  which  the 
author  develops  the  theory  from  his  chosen  starting  point© 

The  user  of  time ^continuous  processes  in  the  applied  fields 
who  is  not  interested  in  the  methods  of  proof  may  still 
appreciate  having  a number  of  important  definitions  and  results 
conveniently  gathered  here  between  two  covers. 

Finally,  it  is  hoped  that  the  publication  of  the  mono- 
graph will  stimulate  further  expository  efforts  in  the  import® 
ant  field  of  time -continuous  stochastic  processes ^ and  that 
in  particular  the  day  will  come  a little  sooner  than  it  other® 
wise  might  have,  when  a comprehensive  but  readable  textbook 
en  this  subject v using  the  measure®theoretical  approach, 
appears  in  the  English  language© 


Jo  Ho  Curtiss 


National  Applied  Mathematics  Laboratories 
National  Bureau  of  Standards 
V'ashington  25  * D*  G© 

October  1951 
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INTRODUCTION 


The  study  of  stochastic  processes  Is  becoming  increas:  gly 

' Important  in  nany  branches  of  science  and  accord5.ngTy  the  r th®~ 
matlcal  theo::rT  of  stochastic  processes  has  progressed  rapi  j 
during  the  last  two  decades*  This  rapid  progress  has  resul  ad  in 
a large  diversification  of  notation  and  terminology  which  ; keg 
It  difficult  even  for  a mathematician  to  inform  himself  on  he 
subject*  It  seemed,  therefore,  advisable  to  bring  together  under 
a unified  terminology  and  notation  some  of  the  basic  definl  ions 
and  results  of  this  theory*  The  viewpoint  taken  was  that  : the 

mathematical  atisticiar^  and  the  stochastic  process  was  a : ord®> 
ingly  defined  as  a family  of  distribution  functions  satis  Ing 
certain  consistency  relations.  It  was  on®  of  the  goals  of  be 
present  monograph  to  develop  the  theory  of  stochastic  prcc  scs 
from  this  viewpoint  with  as  little  appeal  to  abstract  meas  e theory 
as  possible*  l.n  most  practical  problems  information  about  sndom 
variables  can  be  obtained  only  in  terms  of  their  joint  dls  ibution 
function*,  and  f t is  the  opinion  of  the  author  that  a treat:  e on 

stochastic  processes  will  be  most  useful  to  the  statistic!  if 

* 

the  definitions,  theorems^  and  proofs  are  given  In  these  tc  &«»  It 
is  in  many  cases  almost  Impossible  to  trace  a result  to  or  par® 
tlcular  autho  % and  it  was  therefore  decided  to  omit  refer  e©i 
altogether*  This  does  not  mean  that  the  author  claims  ere:  t for 
any  particular  result.  To  the  author's  knowledge  only  the:  em  7 
of  chapter  1 and  most  of  chapter  3 are  new*  (After,  complc ; on  of 
chapter  3 the  author  was  Informed  by  H*  Rubin  that  some  of  he 


results  of  this  chapter  had  previously  been  obtained  by  him  ana 
L*  Savage,  but  their  results  were  never  published,,)  In  hi 
sentation  of  ie  theory  of  stochastic  processes*  as  well  as 


I 


chapter  4,  ti  author  has  followed  the  presentations  of  M« 
given  in  Paul  ^evy’s  book  on  stochastic  processes  and  in  M 
paper  ”0n  set  of  probability  laws  and  their  limit  element 
(University  c California  Press,  1950),  respectively*  In 
treatment  of  hunter  data  in  chapter  5 the  author  hac  usee 

Feller1 s appi  ich  and  his  masterful  presentation  In  the  Co 

# < / 

Anniversary  v;  lume*  The  treatment  of  the  Omstein  Uhlenbe 
process  in  cl,  :>ter  2 follows  & presentation  given  by  J*  L© 
(Anno  of  Math  , Vol©  43,  No*  2)* 

My  than!  are  duo  to  Dr*  Eugene  Lukacs  for  his  v&luab 
help  in  prep*  ng  th  ? final  form  of  the  manuscript  and  to 
P.  Mo rand a wl  read  ;he  proofs  and  prepared  the  Index*  I 
wish  to  then}  Professor  M.  Loeve  for  many  helpful  discuss! 
the  subject* 

H*  B*  MANN 

Ohio  State  Ur  varsity 
May  1951 
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Chapter  1 


FUNDAMENTAL  CONCEPTS 


la  Random  variables,  We  consider  a finite  or  infinite  set  of 

r*  xnsjw.  r-~~'-.xc. '•jtarjazrcT* ttrzJ -aa?  .Mfen.i  jcjouts*.  xxvwistt* 

symbols  (x*y5odcs)  such  that  to  every  finite  set  of  symbols 
*■>*••***.  there  is  defined  a light  continuous  distribution  function 
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called  the  probability  of  the  ©vent  x^  < a^,  ,*dfx^  < a^  0 

The  distribution  functions- of  th©  family  given  by  (!el) 
satisfy  the  following  equations? 
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where  1*  is  any  permutation  of  the  numbers  lj>2f00o*&j 
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The  symbol  x-  is  called  a random  variable,,  For  every  Borel  set  A 
in  the  n-dimensional  Euclidean  space  we  define  the  symbol 

P[(x-f>  0.,,x  j cAL  called  the  probability  that  the  "point” 

in 

{x^o0str  5 lies  in  A by  the  equation 
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If  gif,  9 0 * J is  a Bore!  measurable  function,  than  we  can  define 

in  M 

a -new  random  variable  glx^© w * 0*x^)  by  the  equations 

■'  £7  vr  ^»^i»oeo^a)  = Pig  < &9  yX  < ^teo»  3L,  < ^ ) 
j w " 4 • 1 1 -a  m 
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We  now  consider  sequences  {x^}  of  random  variables,,  The 

« 

notion  of  confer  gene®  of  such  a sequence  can  be  defined  in  various 
ways,  Ir  our  representation  of  the  theory,  of  stochastic  processes 
we  shall  hcaaver  use  mainly  the  following  definition,, 


S 0 Convergence 
for  ©very  £ > ' \ q 


A ©equenoe  {x  } will  b©  called  convergent  if 

V 

> O there  exists  an  j such  that 


U04J 


for  n > !(&,  and  all  h*  If  there  exists  a random  variable  x such 

that  11m  Pllx  -x|  > ©)  =•  0 for  all  s then  w©  shall  write 

n-*co  3 a j - 


plim  x*-  x and  say  that  {j;  } converges  to  x ©r  that  x is  the 
n~$>oo  a & 

probability  limit  of  the  sequence  {x  }«  The  convergence  defined 
above  L 8 U£i-,al2y  termed  convergence  in  probability*  This  definition 
can  be  extended  In  an  obvious  manner  to  random  vectors,, 

We  proceed  to  formulate  an  important  property  of  convergent 


sequences 


T/it'  ■ n . 1 f ; t rx  } “bo  a sequence  of  random  variables.  There 

y|  • w 


•inia-ra  a random  variable  x such  that  plia  x & x if  and  only  if 

n~>oo  n 


ta©  aequeno©  {%  } converges,  Moreover,  if  F 


xnyi,**y« 


are  the 


dlefci/lbubioo  in  etlaaa  of  x (n«*l,2,0.#)  and  y a#*y  then 

X m 

Um  ? „ ,.  :,  Ea:v . ...  for  all  points  (t,b1(  ) for 

n*>co  -"n^l 9 5 ■ : x ® 

the  ft-natlor  11. . , v.  (t5b. , ec  e,b  ) is  continuous  in  t0 

3 « * * # _ •*  “ 


which 


'Phccr-jjj  iol  gives  a condition  for  convergence  in  probability 
similar  tc  t--ucty*s  criterion*  This  condition  was  first  established 


by  £«  Slut  eh 


; r.:  - - 


« 


rgi 

Wc>  proceed  to  prove  theorem  Id.  As  a first 


step  w©  sssune  that  the  sequence  |x^|  is  convergent  and  show  the 

existence  of  a random  variable  x » plim  xH*  In  th©  following  we 

114  co  n 

write  for  abbreviation 


g {q)  F 
c'«*  ' 2’ 

L j,  & 


*.7,  ***et5a) 


We  ilrat  pie*'©  the  following  lemma* 


LI]  Itetror  > ,5-89  (19&5)  j 0*  Ee  Aoad0  Sei*  Peris,  187,  370-372 (13 £8). 

I 2]  The  p2T- k‘?  of  theorem  lrl  may  be  skipped  in  a first  reading 
without  affecting  the  understanding  of  the  rest  of  the 

monograph , 
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i25£!L_i*JL  ij  a^y  positive  numbers,  then  £02*  sufficiently 

. rge  n and  all  h > 0 

' 9^)  En(*+S)  * n > Sn4,h(o)  > g.Jo-6)-  sj 

r all  c , 

For  abbr aviation  we  write  for  any  ©vent  & 


pb^S)  p y,  < ? « • » * y < b ) 


k " * 1 ~ll  m — & 

> that  In  particular 


-Vxn  i')  = V0) 


ten 

bUn£°*s>  *VX»£8*6’  *a*h<  «)  2:  Whs°-  >n*h~z3i  Ss> 


laoe  tho  set  of  points  <x  , * ) for  which  ix  - x I > s 

& 1 n*h  a1  " 

^eludes  the  points  for  which  \x  — x ] > S and  x < ©;,  we  have 

n+n.  n n^  3,  *'■» 

b(za*h  ?-  °»  ^xn+h“  V - 51  ^ V^n-h-  °>  ~ p<  lxn+h~  *n<  > 5> 

aae  for  sufficiently  large  a and  all  h > 0 


Sn(o+5) 


£ W0J 


n 


o 


ra&larly 


Ws*  2«a!c's! 


d ( 1 o 5 ) fellows 


n 


For  a sequence  of  functions  If  (t)  i we  shall  write  Lim  f (t) 

1 & i n^oo  n 

:?( t ) if  li^fn(t)  s f(t)  for  every  continuity  point  of  f(t)w 


Lemma  1 n3  There  exists  a non- decreasing  function  g{a)  such  that 


e 


© function*  g (e)  are  non  decreasing  and 

n 

eorem^  tiers  exists  a ^subsequence  g (c 

ni 


hounded  hence  by  Helly 

J suoh  that 


6) 


f4S>sntl°J  * «l°) 


ere  g(e)  ie  non-decreasing* 


Let  t b©  a continuity  point  of  g(o)0  Fix  q>0  and  choose  6 

positive  and  arbitrarily  small  and  so  that  t + 5 and  t-5  are  con- 
Inuity  points  of  g(o)  and 

g(t*3)  - g(tj  < jj  , g(t)  - «(t-5)  < 55  „ 


? sufficiently  large  and  n we  then  have  by  Cl  5) 


«»,  <*♦&)  ij>  «a(t)  > g (t-5)  - ij 

”'4  ^ 

1 therefore 

1-  JJ  > g^tt)  > g(  t-'ft)  - IJ  e 

y choice  of  5 w©  have 

g(t)  >2Xi  > gnlt)  > g(t) 

• ij  san  b$  made  arbitrarily  small  for  sufficiently  large  n. 


Helly  a theorem { seep f or  instance^ VoWidderp  The  Laplace  Transform 

2?)  states 0 If  the  real  non«4eor easing  functions  qlv ( x ) and  th8 
Ltivs  constant  A are  suoh  that  jcu(x)j<  A in  ^ 0,  lf  z9 . 0 «^a<x<b  $ 

. a there  exists  a subsequence  {o^  (x)}  of  {c^Cx)}  and  a non«= 
r easing  bounded  function  a(x)  such  that 

Lim  gl,  lx)  — a(x)  (a  ^ x <.  bi 

i-#QO  aj[  — ** 


. 


, 


. 


. 


/ 


6 


It  than  follows  that 


Lia  g ("t 5 = g(t) 

Ol  G 

and  l©mma  le2  is  pr ovedc 

Wo  now  define  a symbol  x by  the  equations 


?r, 


>K)  - *, 


ooof  «a;  - **y  .y  ,xy4  0 0 0y  ' ”\9  * • 09  wi«ls  **  *1*  • * eS 


g(t  4-0) 


To  prove  that  sr  is  a random  variable  we  have  to  show  that  F 


^3^1*  e osTja 


is  a iigtributioa  function  and  that 


Uob) 


•A«F" •?  C « oFl 


m 


'*  ^1®  O O O 9 ^ 


— ^ 2 V V (tpb«0Ooo*^_ 

jl  m-x 


and 

(109 } .lia  F 


i-~~.  - . (t'gKp  ,9apt  ) — {b,^b  • 0 0 eSb  ) e 

t ^ ^ w -jo9o«^  ^ 2 ® 


That  the  interval  function  corresponding  to  F™  v is  non-negativ© 

o 0<*a 

is  obvious  since  it  is  a limit  of  factions  F~  - - with  this 

nw  x 0 * 0 * in 

property,,  Wa  therefor©  have  merely  to  show  that  (108)  and  (109)  hold 


and  that  F_„  _ tends  to  tiers  if  any  one  of  its  arguments  tends 

z!x c o ®ym 


to  ~co 


Q 


ffe  lave 


a 


( tg  tf  J § 0 o <?  0 ^ S r w ! ?v  4 S 


m 


0 - ?xnti . . .yj  *'  bi»  • • • • V £ V,(V 

M hence 

0 < F. 

o that 

Um  ._  F 

-73 

Furthermore  from  l 1*5)  for  arbitrarily  small  ^ and  all  t 


yi  4 


(t,b,,...,b  } ~ 0 


^ 31  ^ 5*  C I*  ^ 

x ^ v~m 


b.-*-ao  xy.  . J4y_  - m 


wxMy/. . . y*  * * °*  V • * * * *»  * ''  r*  ™ "*y,...y  t',*V  •••***)  ^ 

nim  4 m 


’or  sufficiently  larg©  n uniformly  In  ta  ffe  therefore  have 


lim  F 

5*°>  «* 

Furthermore 


t*^ °oo  © © oy^ 


( ^ls  0009  *\>  } ~ o 


0 - ly. . , .y,  *•  V 0 "’Vi1  - \j  . . .7  ■ 

ii  1 **  1 ® 


< 1 - Fw  (fcj  | 


ym  a 


if  we  let  first  &-t»oo  and  then  to  ~$»oo  we  obtain  (106  | 

m 


To  pro to  (109)  we  choose  a continuity  point  © <t 

F»y  oooV  (t„bA...ba)  »o  IMS*  that  for  fixed  5 asfi  q 

1 "a 


%b&  also 


; y V ^ ® **’’'**  * ° ° «>t'  j ss  ,-:c 

AnJ,}oo#,i  4 » 


’W  V ^ ^ 1 9 © o o g>  0=3 


F 


'n^i° 3 c^m 


t 5j>  tp  o o oS> 


F«-  « ■'  .9  o o o 5 ! 

p?  2S  ® ° c ^ ^ ^ - 


is 


where  0 < e <,  1 and  0 < ©*  < 1 


. 


8 


t ; n ;;  Ski  a into  i . 0)  .va 


9 O 


■%  y 

nohri 


j i *1?  b,  p 

*J&  A 


e £ 


iO 


> 


0 0 9 


yJ-V 


€90 


» 


/ ie  tf*  vg  h-s»QO  and  oonaiderlng  that  f may  be  ohosez  arbitrarily 

:aXX  we  obtain  (109J0 


file  proceed  to  pro that  plim  s 

n«*>  oo  a 


x,  tfe  represent 
domain  ]y®x|  > e by  th©  sum  of  a denumerable  number  of 


Intervals  whose  comer  points  are  continuity  points  of  F for 

xnx 

Ll  a„  To  do  this  we  remember  that  there  can  be  at  moat  a denumerable 


number  of  points  with  -«  positive  probability  in  the' plane 9 so 
we  can  construct  an  interval  netting14^  W’aioh  avoids  these 
points^  Since  the- set  ( y ~ x j > e is  open0  every  point  of  this  set 
1*  Of  n ; ined  in  one  inte  rval  of  this  netting  which  H entirely  con- 

- ; - - u L'or  •'<  we  have  mere ly  to  tal  s-  Ll  the  inter  vale, 

of  cur  netting  which  lie  entirely  in  the  set  Jy~  *0  Thereby 

' ■ 9?  *d  :9.  sue*;  h • shat-  i,f!  4..  0I  are  two  sue*  intervals  and 


_ Ixn-ivs  lei  ■;  - j:  ftjiu  v , r.i hi /ides  t .a 

a«-  nto  £3t-s:;  ••.  « is  called  a ue&h,  The  area  of 

& / these  reo^a,-  -lea  4s-  called  the  gcdulue  • f the  mesh0  A 

■ ■>■*■■  e {11^}  of  -3  .-has  mo h that  id.  is  a re  inemeat  of  :ji. 

2’.;'?h  that  the  oduXi  sa|  converge  to  aero  i-  call ed  a netting . 


■ 


then  only  Ij_  Is  chosen  for  our  interval  covering* 

Let  Ij , Ig,  B a 0 he  these  intervals  and  denote  by  P . (I^j 

the  probability  that  the  point  (xn*y)  will  fall  into  the  interior 

either  or 

of  the  interval  lv  or  onAits  right  etsd:  upper  boundary0  Then  for 
sufficiently  large  n and  arbitrary  ij 

p(iwxJ>£>s  Fxnxnth  lI*> 

for  all  hc 
Furthermore 

P(|Vxi  > ■ ) s |i>XnX  tlk) 

Both  sums  converge*  From  now  on  consider  n as  fixed0  Choose  I 
so  that  for  some  vj  > 0 

JjfVl  PV  (Ik>  < • 

Next  choose  h so  that  for  the  first  N intervals 
! Uk)  " ?xnx  (I*,!  - f~ 

*'*  o 

Then 

Pll*n“xl  > e)  s pV  (Ik>  i f Pxnxn4h  (Ik>  2n<  3n  . 

t 

Since  2|  was  arbitrary  ^n^P(|xa~x|  > e).s  0 or  plim^x^ax  0 

[Th©  relation  glim^x^x  also  follows  from  the  fact  that  the 

characteristic  function  of  x-xF^n+h  00nV®rg@8  to  character- 
istlo  function  of  xn=»x  ] 


o 


I 

I 


On  the  other-  hand  if  pllm  than  tor  sufficiently 

large  n and  arbitrary  q 

P(  |x  •xOT|  < £)  > P(  |x  ~x|  < £ and  lx  . < £ J 

! n*h  n!  w 9 — f n 1 - *x  1 n*h  * — jt  1 

# 

> p<  |xn-x|  < «j-p(  |xn+h=.xj>  | ) > i-n  o 

Hence  the  seg,uenoe{xnl converges  and  theorem  101  is  proved0 

30  Stochastic  processes,  it  set  of  random  variables  x^  where  t 

is  chosen  out  of  some  set  of  real  numbers  is  called  a stochastic 

process # Lf  the  '•eet  of  indices  t is  an  interval  then  the  stochastic 

process  is  said  to  depend  on  a continuous  parameter 0 Such  a process 

is  called  continuous  in  [a*b]  if  for  every  sequence [hfo^tlth 

11m  h.  « 0 • plio  x.  . s x,  for  a « t < by 

1«^00  A l-»0 0 **  ~ — 

The  expression 

^oo 

f t dF«(t)  s Ely} 

®oo  * 

is  called  the  mathematical  expectation  of  y0  The  expression 
E{[2  •Elx)][y»E!yj]}g  gxy 

is  called  the  covariance  between  x and  y0  The  covariance  between 

and  will  be  denoted  by  cr^  ^ and  called  the  oovariano© 

1 3 1 3 


function  of  the  process 0 


11 


4*  Convergence  In  the  mean,,  A sequence  of  random  variables  {x^} 
Is  said  to  converge  In  the  mean  to  a random  variable  x in  symbols 


Xoiom0  xas  x if 
n-^oo 


'■1,18) 


lim  E(x--x)2  5 0- 


shall  now  prove  several  very  useful  lemmas  on  oonvergena® 
in  the  mean  and  convergence  in  probability,, 

Lanuaa  1„3.  If  thsa  gUm  xnsx  „ 

This  follows  Immediately  from  Tohabicheff "s  inequality,, 

Lemma  1,4.  If  l„i0m„  x.sx  then  lim„  E(x„)  s E(xj  0 

n«^oo  ^ n~>oo  u 

Sine®  E(x  *»xj2  < e for  sufficiently  large  a we  also  have 

e > E(x  -x)2sa|  .x*(E(xn-x)]2  > [E(xn)-E(x) ]2  0 
ju  n 

Lemma  1B5Q  If  {y^  is  & sequence  of  non~neg&tiv©^ D-'  random  variables 
and  if  glim  yh  g y and  E(yh)  <M  then  E(y)  < M 0 

Under  the  conditions  of  the  lemma  end  in  view  of  theorem  101  w© 
have  F^it ) s Fit ) where  and  F are  the  cumulative  distribution, 

functtonsof  and  y respectively*  Suppose  E(y}  > M then  there 

exists  a continuity  point  A of  F(t)  such  that  ft  dF(t)  > M * 

A . A A ' 0 

However  f/dF^(  t)  < * and  31m  ]%tdFj1(t)  s ftdF(t)  B a contra^ 

0 a 0* 

dlotionrt 


[5]  1„9„»  if  £(yh<  OJ  S 0 


o 
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Lemma_i^60  The  sequence {x^}  conr&t gas  in  the  me&n  to  a random  vari- 
able x if  and  only  if  to  every  £ > 0 there  exists  an  M such  that 

U0lS)  Etx^-x^}2  < e for  all  mpn  > IS  0 

Suppose  first  that  there  exist  a a random  variable  z such  that 

X^i.mo  xn % x c Then  for  sufficiently  large  n and  m and  arbitrary  £ 

n*<*w 

B(xa-xi2  < e j,  E(xm-x)2  < e 0 
But 


E(zm“X}2  ♦ Elxn»x)2  -2E[(xffl-x)(xn-x)] 
and  by  Schwarts's  inequality 
f E{xm~x)Un-xj J <^/B(xm*x)S  e(xh«x)2  < e * 


Rena  a 


^^nTxn^  .«  the  other  hand  from  E(xm^xnj-  < £ it  follows 

by  Tchabioheff's  inequality  that 

±*l  \x  *xw|  > tc ft  ) < 1 
1 a 23 1 - ' ~ T2 


Thus  plim.  x„  sz  exists  by  theorem  l0le  It  follows  also  that 

2£«f>  GO  ** 

plim  (x  ;m*x  )s  • (x  «x)2  and  thus  bj?  lemma  105 
h-**^cd  a u 

S^-*)S  < e “4  i'lS*  *»*X  « 

lemma  l.?0  If  l0l0m0  x_sx  »•  l^i.ffic  y„sy  and  If 

n«^a>  n^co 

E(x2)  j,  E(y2|  exist  then  lias  S^.yJ  s ^y)„ 
n **  n***©  ** 


' h rtapoot  to 


show  first  that  £(x'r}  , E(y^)  z.ie  bounded 

n n 

By  virtue  of  lemma  le6  there  exists  an  m and  an  M such  that 
2^3‘-!a*xn)2.  5 14  for  aii  & From  the  almost  trivial  inequali- 

y i 2[la-b)2*b£]  we  see  that  for  all  m 

E(*q)  £ StEiXa-X^)2  ♦ E(x2)]  < 2[M  * E(x2)] 

. rom  this  inequality  it  follows  that  E(x2}  is  bounded  for  all  m 
nos  ' z(xayn)  | < <^{x|}s^y2)  it  follows  moreover  that  E(xy|  exists 
nd  fur  her  more 


ECx^y^^xy  ) | ® E ^n  * ) * yi*£»x)]j 

<.  7E(x|)E(y^y;2  * ./E(.y2)E(xn«x}2 

: iy";  exists  by  lemma  1,5  and  Efr2)  iS  bounded  and  since  E(yn»y)2  e 

£ *^2 1 converge  to  z s ’o8  the  right-hand  side  of  (I„l4|  converges 

vi  ioh  prc?v6'$  ler;jia  5l0^0  ^emma  107  may  also  b<g  written  in 

he  form 


;X015J  E{U0i0m0  xn)U0i0m0  ynf}r=  He  EU„yJ 

xn)U,i,«0  yaJ}S  Urn  a. 
n—*  co  n-^^>  n— »co  '«*n 

Corollary  to  lemma  1„7„  She  sequence  {x£}  converges  In  the  mean 
ano  o :ly  if  li*  B(x^)  exists  irrespective  of  the  manner  in 


ioh  n 4nd  :a  tens  to  infinity 0 


- ] Thig  ie  seen  if  we  determine  N --  aooordin^  to  leama  lc6  — , so 

£ 

^hat  &{  < e for  m,n>  H and  then  take,  for  a fixed  m>BP 

a * aax{E(xm-x1)2,  E(xk.x2)2,  ...„  E(xffi-xK)2  . e} 


I 

■ 


I as  &•**•**»*  — a - J * 

■-At  **■  «<««“  »*  * * (**-«*  ' 
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From  lemma  X07  it  follows  immediately  that  th©  condition  is 

necessary*  I'o  show  that  th©  condition  is  also  sufficient  we  aseuma 

that  lim  E(xnx  } exists  and  is  independent  of  th©  maim©?  in 
<=£*&•  ^ *** 

m-f>oo 

whioh  n and  m go  to  infinity*  Then  lim  E(v3L)2:0  fetnce  it  is 

a— ® 

ffi-^03 

possible  to  find  for  ©very  & > 0 an  fi§K(ej  such  that 

E(xa«»x  )2  < s for  ng,  m > I „ «Ve  see  therefore  from  lemma  Xes  that 

X0i*m0  exists*  This  corollary  is  due  to  m0  Loev@0 

n—*©  - u 

5C  Differentiation,,  In  order  to  be  able  to  define  derivatives  of 
stochastic  processes  w©  have  to  extend  the  concepts  of  limits  in 
probability  and  limits  in  th@  mean*  Suppose  that  for  @v®ry  h in  an 

interval  (a#b')  a random  variable  is  defined,.  If  for  every . sequence 

{hfl  with  lim  h,  s a 9 plim  xhjt  §i  exists  then  we  writ® 

1 %l  i.-*oo  i 1—  >»  hi 

glim  s x 0 In  a similar  manner  we  define  x..vg|0i^m0  xj^ 

The  process  is  called  differentiable  at  the  point  t if 
glim  xt»froS:t  z exist®*  Th©  stochastic  process  is 

called  the  derivative  ©f  x^  0 

In  the  following  we  assume  E(x^)sO  a Th@  modifications  ©f 
©ur  statements  for  th®  case  s(x^)  ^ 0 will  be  obvious 0 
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60  Stochastic  processes  of  second  or d er n A stochastic  process  x^ 
is  called  of  second  order  if  for  any  values  t7^t^  the  covariance 
ctlt  exists.  The  process  x.,  ia  called  differentiable  l.i.m.  if 

l.i.m.  xt*h”xt  __x*  exists. 
h~*»  0 S“  * t 

Theorem  1.2.  fleoessary  and. sufficient  that  the  proeesg  x ia 

t 

differentiable  l.i.m.  is  that  the  limit 


U016J  lim  t-T  - Z* 

k-£o  1 


exist 0 The  covariance  function  at  t is  then  twioe  different!  *- 

3S\t.  3St 

sdjl®  ana  . x 3 1 B Moreover  x!  is  a stochastic  process 

Stl3t2  3"23tl  ? 

e‘'°t1t2 

of  second  ord©r  and  its  covariance  function  is  • 

^ 2 
3^t*t  ° 


Th®  cov&rianc©  between  x^  and  x^  is  given  by  — “pr 


Proof:  Consider  a sequence  of  difference  quotients 


c 


W©  have 


gt*h.t*k~  gt+h.t  ~ gt. t»k** gt.t 
hK 


and  by  the  corollary  to  lemma  1.7  the  relation  (1.16)  is  necessary 

and  sufficient  for  l.i.m.  ^t+h  st  _ to  exist.  £^3  ex- 

h-*-  0 h 


pression  Z^  t in  (1.16)  is  called  the  generalized  second,  derivative. 
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we  moreover  hav®  by  lemma  104  E(x^j~0  since  Etx^J  ^ Elx^j 


■s  0 0 Furthermore  by  lemma  1#7  cr~ 

t**t 


exists  and 


1„17 1 a*.  =•  11m  E lx*  * ^t^h^tl  _ lim  at*lu  t*“att*  _ 3<ytt 

1 xt*xt  h^O  [*  — Yj  * h"»0 g ' - ~WT' 

Thus  gott*  exists A It  also  follows  from  lemma  ln7  that 

* • 

exists  and 

xtxt* 


a,-?-*  sr  lim  E[xt*e*h~xt  « xt*$*k  aXt* 

l **  fct8  — e— 


= ^ qt«h.t**k~  qt»t*«k~  qt»hat**gt.t*  _.  Z^* 


It  easily  follows  that  c^.*  is  twio©  differentiable  and  that 


S^tt* 

IFsT" 


&2gtt* 

3F"W" 


© 


It  is  well  known  that  the  generalized  second  derivative  of  any 

function  f(x,y)  exists  if  i'f  exists  and  is  continuous,, 

ax  ay 

Thus  we  have 

Corollary  to  theorem  ln20  If  x^  is  a stochastic  process  of  second 
order  with  covariance  function  a***  and  if  3 G%%  exists  and  is 

u at  at  * 

continuous^ act  than  x-£  exists  lei.a,  and  its  covariance 

function  is  ^gtt*  _ 2,  . * e 


The  oor Hilary  to  theoi  Ic2  giv<*e  a coiv anient  raebl  >1  to 

:i  ki  t 'O variance  : u*  Lo>  oi  x!  il  x*  exists  1„:  mc 

\ 

Int  grationQ  Let  x^  be  a stochastic  process  defined  for 

< t < b , tie  subdivide  the  interval  from  a to  b into  n parts 
y means  of  the  points  a «:  t )9t^,  09(,,tn  ~ b aad  put 

ax(t  ) - 8 0 The  number  6 is  called  the  modulus  of  the 

subdivision*  within  every  interval  t.  , < t < t we  choose  a 

'aluo  t ‘ and  fora  the  sum 
x 

i 7 ^i-1 ) 

is  a random  variable,,  flow  consider  a sequence  {SM}  of  sub- 

t vie ions  Sra  with  moduli  such  that  lim  --t  0 c l*et 

m m m *>&  m 

■ *&  be  the  random  variable  oorraspon&ir :$  by  {1018}  to  the  sub- 

ivlsio  i and  some  she  ice  of  ;he  ta  , £ then 

lim  X.I rnj  ~ X exists  and  is  equal  for  all  sequence  {S’*  with 

ol-*q& 

modulus  converging  to  Zi&o  and  all  choices  of  t*  th  a X is  called 

he  Integral  of  x.v  and  we  write 
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[7  ] 

Strong  contl nnl ty 0 In  th  e 

the  event  that  the  relatione 


following  we  denote  by  £(89e,S) 


lxtr  x 


< £ 


are  simultaneously 


satisfied  for  all  pairs  with  | < 5 

to  a finite  set  S of  points  contained  in  [e=>b1 
is  then  the  probability  that  the  inequalities  ]x^ 


and  belonging 

o £[<£  ] 


are  simultaneously  fulfilled  for  all  pairs  (t^t,  | of  a finite  set 

* ™ 4SU 

S of  points  for  which  Jtj-tjJ  < $ 0 

The  process  is  called  strongly  continuous  in  an  interval 
[a*b]  if  to  every  £ and-  i)  tfcere  exists  a S-Slt^ri?  aash  that 
for  every  finite  set  S of  points  contained  In  [a^b] 

Uc20J  P [g(5ae0sq  > l-n  c 

For  any  stochastic  process  consider  a set  S =*(t^f  0 0 0*t  ) 

where  a < t^  < b ( i = I^^n) 0 We  denote  by  M^g  the  largest  cf  the 

values  000„x^  0 Let  {S^}  be  a sequence  of  subdivisions  of 

the  interval  [a9b]  whose  moduli  converge  to  £©ro0  If 
?y&M&bs,  - ^ab  @xlsts  and  is  th®  same  for  all  sequences  {S^} 

X =4-  itO  J ^ 

whose  moduli  converge  to  zero  then  we  shall  call  K;1>  the  maximum 
of  in  [a^b]  0 The  minimum  m is  similarly  defined. 


concept 

[7]  The  delimit  tea  is  due  to  Pc  levy0 


19 


To  simplify  the  notation  we  also  introduce  ?ab  - Mafo~ffiab 
a Ed  Y&bS  =?  0 tie  next  derive  a criterion  for  the  strong 

continuity  of  a process s 


Theorem  1030  a process  xt  Is  strongly  continuous  in  [a9h]  if  and 

only  if 


(l)  it  possesses  a maximum  and  a minimum  in 

every  subint  erval  [t*t‘"]  of  [a9hj  f 

(ii)  for  every  £ > 0 9 rj  > 0 there  exists  & d such  that 
for  every  subdivision  S ~ (a  « t ^ 0 e G®  t^s  b $ with 

modulus  less  than  o ^ f h ts  i~rue  t’h^t 
do  El)  £ jtjp*  ^ 38  « • ••f11)  5^  ^*”*1  o 

tfe  emphasize  that  (l^El)  means  that  the  probability  of  the  simul- 
taneous fulfillment  of  all  the  inequalities  t < s !is.L,  30e?nj 

* 4 „1  ^ -S*”* 

. , [&3  " " ’ 
must  exceed  1 -*j0 


[83  If  H;i  (lul929  0 6 0»nj  are  n events  then  P(R^t  l9eooj>a) 

means  the  probability  that  all  n evants  occur  simultaneously c 
Thus  PCRj^tsl*  e 0&9n)  > k means  that  the  probability  of  the 
simultaneous  occurrence  of  all  n events  exceeds  kg  this  should 
ba  carefully  distinguished  from  the  statement  P(Rj ) > k9 

( t~X3  0 0 0?n)g,  which  means  that  the  probability  of  the  occurrence 

% 

of  each  single  event  H.s  exceeds  k which  does  not  imply  any 

thing  about  their  joint  occurrence^  We  further  emphasis© 
that  in  conditional  probabilities  the  condition  is  separated 
not  by  a semi col on  but  by  a vertical  bur „ 
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The  proof  of  lemma  ia8  is  left  to  the  reader 0 

Proof  of  theorem  l„3a  Suppose  first  that  conditions  { i j and  (iij 
are  fulfilled^  Let  S he  a finite  set  of  points0  vVe  can  consider 
sequences  of  subdivisions  {Sa}  such  that  each  S„  contains  S 0 
It  follows  then  from  lemma  le8  that 


{Sa}of  subdivisions^,,.,  0<>o8ta)  with  t,,sa  +MMI  „ for  auffioi- 
fatly  largs  a ana  arbitrary  e,jj  we  ha  ’/a  by  (lij 


Ko®  let  t 


(1„22J 


(1*235 


The  relations  t < s , 


isl»2»  ..  „»n  Imply  the  relations 


'9  o o o 8 


n-1  0 Hence  from  (l0£3J  and  lemma 


I«6  it  fellows  that 


t — i-lf  E 
i-lid 


9 © o © S 


n-l)  > 1-ij 


o 


?x 

Any  two  points  tj9t'  with  1 j < i He  together  in  one 

Qy^Ji  (i'2Z) 

of  the  intervale  (t,  „ Henae  the  above  inequal ityAtmplliUL 

1 1 0 24 ) 2 [g  (|9£9S)J  > l-n  „ 

Ehus  (i)  and  (11)  imply  strong  continuity* 

>v©  next  show  that  the  condition  is  necessary,,  We  assume 

that  x,  is  strongly  continuous  and  let  a < £ < H < b0  Wo  consider 
t 

two  subdivisions  S„  and  Sm  of  both  of  modulus  less  than  5 

tei4,  S consist  °f  fhc  f>e.»i-s  o.nd  Sm- 

and  the  maxima  and  CA  *'  " relation 

|M_,  - U'„  | > e Implies  that  for  «Qr  two  points  t9tf  of  3 

abSa  a5Sm' 

we  must  have  J > £ * jt— t \ 5 h^nce  ty  ( 1 0 ^ : l * o ■ 

sufficiently  small  5 and  arbitrary 


(1„25) 


PM%5s  “M5S3  l>£!^ 
n m 


On  account  of  theorem  101  the  relation  (1025)  implies  that 

plim  Jd-rQ  ss  Mgg  exists*  In  a similar  manner  it  is  shown  that 
h=$»cj  & 

7i\m,  exists  so  that  condition  (i)  of  theorem  Xc3  is  satisfied,, 

Hoy?  choose  5 so  that  for  every  finite  set  S of  point  is 
ti$  0oo»tn  and  arbitrary  e9q 


1«  26 


^ } 1 ^ 


How  let  Ss  {a^tof  ooopt^s  b}  be  any  subdivision  of  modulus  leas 
than  5,  {S^}  a sequenoe  of  subdivisions  with  moduli  conversing  to  zero  , 
and  containing  the  points  of  S„  The  relation  (10B6)  implies 

( 1oB7  ) 1*1  tS  *5  » ^ eoo»n)  > 1*^^ 

*1-1  in  2 

How  ohoose  9*  so  that  < S*  < * and  so  that  e*  is  a continuity  point  of 

the  distributions  of  V . is  l8a#  000,n0  It  then  follows  from  ll027) 

ft 

-h  i § » isl^  s®  0 0 0&n  j - > 1 — ij  o 

*1-1 ui  - 


This  completes  the  proof  of  theorem  l03n 

ShooreF.  1q40  t>et  x be  a strongly  oontinuous  proaess,,  than 


IS  X.  - 


21  x*  = 


> x^dx  exists  for  every  t 


dXj 


•t 

ar* 


Proof”  By  theorem  1.3  exist  for  all  pairs  Cj' 

and  we  have  for  every  choice  of  points  axt  „t  ,t  . t st 

0 1 2 • « 

and  h-l  ^ **?. 


(1028J  2m, 


To  understand  this  inequality  correctly  w©  must  remember  that 

^ ^ (i»l#2f  00©«ni  random  variables  and 

"rv?  •'  joint  distribution  is  such  that  th©  inequality  (1628) 

holds  with  probability  onec 


. 


ustufo 

N 

31 

I 

- 

■ 


I * 


I V 


*> 


. / 1 


* 9 ’ 

• 

. 

• 

. 

• 

’ 

. 
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Since  the  process  is  strongly  continuous  we  have  for  any 
subdivision  with  sufficiently  small  modulus  6 


PK,  ‘VW  5 Mb"a)]  * 1 = s>  • 

L 1-1  i I®1  1 ^ 


If  S is  a subdivision  then  we  call  YCSjs^M^  ^ the 

1ct1  i 


upper  sum  and  y(S)s  £m  It  -t  ) the  lowsr  sum  corresponding 

**  ij  t v - A A A 

i®JL  1 


to  the  subdivision  S0  rte  consider  now  a g?®%uene©  of  subdivisions 

{S.}  with  moduli  {5., } such  that 
« 3 


11m  5.s0  and 
$~4*x>  3 


If  Y.^YCS^ji  and  ^sy(Sj) 


SC8  if  m < n „ 
m n 

are  the  corresponding  upper  But  lower 


sums  then 


and  hence  for  sufficiently  largo  n 


PL0  - yn*k'yn  - s(b"a0  £ lo>Jl  ° 

Hence  the  sequences  of  random  variables  {/..  } and  {Ynj  converge 

and  plim  y s plim  Y D From  hare  on  the  proof  of  the  existence 
h-^s©  "o  n~»*co  n 


t 

of  X^sfx^dir  is  precisely  the  same  as  that  of  the  existence  of 
a !" 

the  ordinary  Riemann  integral  of  a continuous  function,. 
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It  follows  also  from  11*38}  th 


tf 


(‘tU)  > J z 


Yi 


Consider  now  th©  quotient 


Xti-  <S£»  X1}* 

h2  bl 

"’"irr^Hrr** 

3 1 


tg 

f xtfi 


t 


1 2 


nV©  ha¥@ 


xt  - xt, 

m.  , < t;E  S1  < M 


1 ■{*  «$=■ 
“18 


zmH 


t-j  t > 
2 


aM 


m..  . <•£,  < M.  . 

*1*2  “ H"  *1*2 


and  sine®  t h@  procst;,  if3  strongly  continuous-' 


• <T: 

&* 


s?,v Mp  “%t. 


Henoo  pH®  m.  , s plim  M,  , 

4:  aiaafe^  »i  »/w  ifcesa^jj  «*8  4; 

2 ,1 


““S’Tt,  wp  j -p 

e&  Cj  <t-.  <*i  .Ci 


1 


and  thus 


plim 

’•r*h- 


«*  ^t' 


A4t 

C«  * j • 


Tills  cempletea  the  proof  of  theorem  i-b4  « 

$©  shall  now  consider  siochastie  proaosses  of  secr.aci  ordCT. 


..  i 

X.  ^ f x.dt  exists 


if  the  Hi ©mann  sums  2x.  w(t4«*t 

C V5  i 


4^*1 


•on^r©3 


in  th©  m®an0, 


25 


Theorem  1050 
The  process 


Let  xt  "be  a process  with  ©©variance  funotion  0^  f 

X 2 

is  integr&ble  X0i0m0  in  [a^b  3 if  mid  only  if  for  any 


o 


t t 

t in  [a^b],  f J t dtj&tg  exists.  The  covariance  funotion 
a a w ,l  * 2 

t 

2;t  t of  Xt  = J[  x^dt  is  given  by 


.1.89) 


tlt2  ” sf 


^1^2 


0. 


4'sidT2 


The  process  X^  is  differentiable  X0i«m0  and  X^ssx^ 
continuous  0 

To  prove  theorem  105  we  apply  the  corollary  to 


if  0a  i is 
*1*8 

| 

lemma  10?  to 


a sequence  of  Kiemann  euas  {2  } with  moduli  going  to 
w©  have 

e‘v«*s  EK?C1 vwzxtt(yvi)]  *f5« yw{1 Wi 


J 


If  21  and  m go  to  infinity  in  any  manner  we  have 


t t 

lim  E(J  J )=  f T of  * dt  dt 
n~& QO  n m a a • *1^0  1 1 


Thus  X0i 0m0  2 « 2 exists 0 
n-^oo  11 

Moreover  if  {2^}  is  any  other  sequence  of  Kiemam  sum©  we 
put  2^s  2a  0 2n  0 Sinoe  2”  ©xists  we  must  have 

2 s X0i0®0  2n  ™ l,ic®o  2 = J x at  ^ X 
n~»®  n n->oo  n 5 
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It  follows  also  easily  from  lemma  I,?  that  J5(x^X+/ } exists  and 

that 


(lo2S&) 


E«xtV>= 

* a & id 


w©  further  have 


Y . '•Xa. 

**.  <0  »* 


Xt  « ?Vxt  4,  , 

t tv  ~Fn 


From  Jl029a)  it  foil ova  that 


.t' t' 


*t»y  x* 


Trrajf  &V "°t t' "0t T *°tt ^'c' 


If  a , is  continuous  then  by  the  mean  value  theorem  of  in 


tegral  calculus  ov  v becomes  arbitrarily  small  if  t* approaches 


t0  Thus  x^ss  Xt"  L0louo  * 


Let  x^  p y^  b©  two  ft 


=■  than  to  every  a 


division  S s Ca^t^t^g  t^  00  0*ta«b)  we  can  form  Riamann-Stislt  Job 


sums 


n 

ilS)»  rx-  (y*  - yt 

1*1  *t  i i-l 


(lo30J 
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If  nov*  for  every  sequence  {S  } of  subdivisions  v;ith  modulfe 
converging  to  zero  piim  xis  )'  exists  and  is  Independent  of  the 


n-^oo 


particular  sequence  {SQ}  and  of  the  choice  of  points  t 
(t^^  < t*  < tjJ  then  we  shall  write 

X=s  plim  I(S  j - S xtd^- 
n-*  oo  n a w 


*Ve  shall  call  X the  integral  of  x^.  with  respect  to  y+c  If  the 
random  variables  XjSnJ  converge  in  the  mean  to  a we  shall  say 


that  k 

^ x^dy.  exists  l0i 0m. 

* t 


Theorem  1^6^  l«et  x..  9 y^  be  two  independent  stochastic  processes 

of  second  order  (that  is  to  say  is  independent  / 

g 

and  any  t)  with  covariance  functions  * Pttf  *®speotively„ 

The  integral  of  x^  with  respect  to  y^  exists  1 ;i  aa  foi 
terval  ra*t]  contained  in  [a;b]  if  and  only  if 


(1.31) 


t t 


JJ 


exist s0  The  covariance  function  of 

xt.  - S xt4^t 

a 

is  moreover  given  by 

h t t J j o <¥* 
*1*2  a a txt2 


o 


The  proof  of  theorem  1,6  is  analogous  to  that  (■ 

1,5  and  is  left  to  the  I'e&dsr, 

By  P(S|E)  we  denote  the  conditional  probability  that  E '.'fill 

r o i 4? 

happen  provided  E has  happened,1  ’ 7 .ia  ha?j  called  a 


tiauoua  if  dim  x 


’S-&Q 


- X 


4*  o 


A process  will  be  celled  uniformly 


continuous  in  [agb]  if  to  every  e > 0 g q >0  there  a .tie 
5? £<,ii  | of  t 3\ii3 h.  tnsi ti 


$e  a 


pl!ztnxtl  & s)^  1’1l  every  |t]  < o(  evs|  J 


Lemma  l09e  if  a process  is  continuous  in  a doted  in.ervd 
[&£>&]  then  it  is  uniformly  continuous  in  ; &s b'j 
Troo^t  Consider  a monotone  decreasing  sequence 

dim  t4»0  c For  every  i consi&sr  the  set  of  points  for  ?hieh 

p(lxu<xtl  > *1  >*« 

c& 

Assume  the  lemma  to  he  false*  then  we  can  construct  i aegu^uoo 

ti<,tg9o,,  such  that  /e/-  ;<,.■««  f '><>,  ■'rj>o 

P(|xt^T"xtJ  >;?£> 

for  some  % <.  t*  * Let  t he  an  accumulation  point  of  & e ecue-nee 

5>o 

{t.}c  Then  for  every  « w-a — ft  we  can  find  a 


to  t such  that 


| > Zt)  > £*}  for  some  j-cj  <>  J| 

1 * i 


[3] 


^02?  vile  concept  of  conditional  probability  the  read  a 7 Is 
referred  to  Kolmogoroffg  Grundbegriffe  dsr  Wahu 5 chain  ~ 
lio hkai t sreefenung,  Chapter  5,  par,  ; end  3. 
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Choo>  e now  |t.~tj  < and  t ^ 30  clos  to  t that 

P(  >*}<!), 

S 

Then 

?(I^+T-Xtl  > Z PM*t.  ♦?***.!  > ge  * !*tl-xtl  i e2 

> p(  >*}>*»  . 

5-3  & 

Honoc  for  arbitrary  &>Q  and  some  e>0a  q>  0 there  exist  values 
£ < h such  that 

p(lxt*T*xtt  > e)  > n 

in  contradiction  with  our  assumption  of  continuity  of  the  process  x^c 

rie  derive  next  a sufficient  condition  for  the  existence  of 
M h f nd  hare  to  aon«lde**  in  this  connection  the  event 

{*  <a  Xi  f0r  k < * 9 xfc  < Xi  f0r  k > Rnd  W®  denote  this 

by  A and  state  the  following 

Theorem  ln7Q  Let  Ss{a  < ti  < t£S>oooy  < b}  b©  a sat  of  points 

in  [a^b]  and  let  xt  be  a stochastic  process  which 
(ij  is  continuous  in  [ayb]$ 

{iij  is  such  that  for  sufficiently  small  t —which  is  inde- 
pendent of  3 — 0 * 

(1Q3£)  > *lAi>  * ■ <xi^nrXt|  : > " >! 

where  K is  a constant  independent  of  the  choice  of  S-Q  Then  and 


mafe  exist 0 


I 


’ 


V 


■ 


■ 


' 
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Proofs  Let  SiS  (t1# .0#ftn)  and  S = ( 0 . c, l£)  be  two  subdivisions 


of  module  less  than  6 and  put  for  eh. or  • ^ = o 

q 


To  every  we  can  find  an  Xj  s y^ 


a 


such  that  t 


J- 


t.  < 6 


For  P(A^J  4^  Q we  thus  have  cm  account  of  lemma  1*9  for  arbitrary 
ij  and  sufficiently  small  5 

(1,33)  P(Ak,zlc-yk  > e)»P(Ak)P(xk-yk  > eUfc)  < , 

If  P(A.j'sO  the  inequality  (1033)  i3  also  valid*. 

Denote  by  3^  the  Joint  occurrence  of  the  evenia  A^  and 
xk~^k  < 2 an&  1*^  B be  the  event  that  at  least  one  of 


the 


©vents  Bv  occurs*  By  definition  .A  Implies 

■&  K. 


x.  *®  M , _ so  that  B implies  the  existence  of  soma  y, 

K aoo^  “ jf 

such  that  y,  > M . 0 - £ 0 From  this  it  follows  in  turn  that 
i£  “»  aob| 


M. 


> M, 


~ s 


abSj  £ ~ ° 


Therefor©  we  see  that 


•^afcSi  •-  MahS|““  £J  £ p^Bi  S ° 

From  (1033)  obtain  easily 

P(Bk)  = P{Ak)P(xk-yk  < e|Ak)  > (I-K^)  P{Ak) 


o 


The  ©vents  exclude  each  other  a n£  exhaust  all  the-  poesi 
bilitiee  so  that  by  adding  these  Inequalities  we  obtain 


Therefor©, 

(1.34) 


2P(BV)  > X-Kir, 
k *' 


f ^MabSj  MabS^~ 


® | ^ 1 o 


Similarly  w©  obtain 


Cl034a'j  ^(“abSj  £ MabS,~E^  £ 1-1  ^ • 

hence 

U.35)  PM\bS1"  MabS  J i e>  >1‘'2K’}° 

The  existence  of  Mafe  follows  easily  from  (lc55)  using  i he or am  1, 

and  th©  existence  of  is  proved  similarly 3 


CHAPTER  2 


SPECIAL  PROCESSES 

/I  A email  partial©  suspended'  in  a gas  is  subjected  to  a contin- 
ual  bombardment  by  the  molecules  of  this  gas*  The  individual  impacts 
imparted  by  these  molecules  are  small  compared  to  the  mass  of  th© 
and  the  n.imber  of  impacts  per  second  is  very  large*  The  im- 
pacts are  received  from  all  directions  and  are  randomly  distri— 

t 

buted*  Moreover , if  w®  neglect  the  velocity  of  th©  particle  : 
itself*  which  is  small  compared  to  th©  velocity  of  the  molecules* 
th©  distribution  of  these  impacts  at  time  t will  be  independent 
of  th©  momentum  of  th©  particle  at  time  t*<  t 0 If  we  denote  by 
the  momentum  of  th©  particle*  it  will  therefore  be  reasonable 
to  assume  that  x^  is  independent  of  x^#  for  t*  < t 0 The 

motion  of  the  particle  is  called  the  Brownian  motion* 

The  momentum  of  th©  particle  is  a special  example  of  a more 
general  type  of  stochastic  processes,  called  Markoff  processes, 
which  satisfy  for  t^  < tg  . a<  tn  < t and  i > 0 the  equation 

P$Xt+T  ^ AlZt1»»«*»Xl!  »Xt)-ptxt*Ti  AlXt>  • 

In  words,  the  conditional  distribution  of  (where  ? > OJ, 

given  the  values  of  , # 0 0,xt  ,x^  (where  t^  < tg  <oe#<  tn  < t) 

is  the  same  as  the  conditional  distribution  of  6iven  2t 


© 


- 


More  conversationally  speaking*  if  the  present  value  of  x* 

is  known  the  distribution  of  any  future  valuer  ie  independent  of 
the  way  in  wbioh  the  present  value  was  reached 0 

In  our  special  case  of  the  motion  of  a particle  suspended 
in  a gas  wa  shall  make  the  following  assumptions  about  its  mo- 
mentum o 
Assumption  Xc 

r-uMXK^a* 


where  e*  ^ 1®  a random  variable  with  moan  sera  and  is  indepen* 

t jT 

dent  of  li,  and  also  of  s * * if  the  interval  a (t^t+xK 

i*  w jV 


(s„l) 


Assumption  2 


o 


The  distribution  of  e*.  depends  only  on 


o 


and  hence 


From  (202J  and  aseumption  3 it  follows  by  a well  known 

£X01  , 

theorem  taat 


(2.3) 


C2~  0% 
T 


where  o 1 a some  positive  constant 0 Thus  x^  converges  to  x^ 

in  t*h&  mean  with  decreasing  t and  the  process  is  continuous 
l0i.0m0  ^ Suppose  further  that  x0~  0 c It  follows  then  from 


(203)  and  assumption  1 that 

t <£,-  •* 

(2.4)  i 

l°xtxt^=  °K!xtn*Vxtl)s^s  ot  - 

The  assumptions  1 to  3 define  fin  important  class  of  Markoff  pro- 

rii  ■»  di 

ceases*  sometimes  called  differential  processes*  1 J "in  order 
to  define  completely  our  mathematical  model  for  tha  Brownian 
notion  we  must  also  take  account  of  the  fact  that  we  regard  the 
impacts  from  the  molecul.es  as  coming  in  a continuous  stream  so  that 
large  changes  of  the  momentum  in  a short  time  interval  became 
much  less  likely  than  small  onesc 


[10]  If  a measurable  function  f(x)  satisfies  the  functional 
aquation  f(x+y)s  f(x)  + f(yj  then  f(xj=f0x  c Proof  of  this 
theorem  may  be  found  in  H0  Hah nf  Theorie  hex*  reellen  Funk- 
tionen*  Frster  Ban&9  ppc  s&l-Sg  J0  Spring&r*  Berlin  (192lJc 

[11]  we  distinguish  differential  process  from  "general  differ- 
ential processes71  (Chapter  4)0 


o 5 


W®  therefore  impose  th£  following  additional  condition* 
called  tbs  Lindebarg -condition*  on  the  distribution  functions 

*Tu>  ot 

Aaavaaptlon  4 (Lind abacs  oonditloajg 

Post  sufficiently  small  X and  arbitrary  P > 0 „ ij  > 0 


12.5) 


f a2  dP  (.a)  <.  no2 

v T s 4 T 

las  >p 


Condition  *{205)  zaay  perhaps  bast  be  understood  if  we  discuss  an 

important  case  where  it  is  fulfilled G 

❖ oo  e. 

-Theorem  2„1„  If  o2  = f ^ d?  (a)  exists  and  if  P{_^5  < a) 

£ -00  * 

-s  Flaj  is  independent  of  % then  fulfills  the  Lin&eoerg 

condition  (£05J0 

ffo^'Tv om  the  conditions  of  the  theorem  vie  have 


FTlaaT)=*F{a)  » Fjaj  ~ Fia/ej  0 

Hence  for  arbitrarily  small  p > 0 and  ^ > 0 


? a"'  arT(a)  * $ *■*  dF(a/ffT)=  a2  f &g  dF(JL) 
!&!>/>  W>f>  ltl>js*x 

■s  a2  j*  y2  dF(y)  < D,a2 
>1  I >/>/«*• 

vz  for  sufficiently  email  » since  CsOi  « 
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'M  next  prov© 

Theorem  20Bo  If  e*.  • fulfills  the  Lind ©berg  condition  then 

.mi  ■i.ri^.af  i i |JU  <W||  (i  «0  % 


uaJ^L’ 


is  normally  distributed  with  variance  q%  0 

as  long  as  we  consider  only  the  distribution  of  there 

will  not  b@  any  danger  of  confusion  If  we  writ®  e for  et  0 
nil  a shall  uae  the  following 
Lemma  2pjLs  Let  XgPooo£>  b©  independent  random  variables  witii^ 

2 

distribution  functions  F^oco?  respectively  and  a^ri4,  )^i 

then  to  every  8 there  exists  a p and  a*s^  such  that 

a 

* . < a % «-  i 


2^oflc^<  a) 


■a> 


@~X  ^ dxf  < 5 


whenev©r  for  all  k 


jxg  dPk(x)  < Jjof  ,c. 
>P  & 


A proof  of  this  lemma  can  be  found  for  instance  in  Khintchine®  ® 
KAsymptoti@eh@  Gesets©  her  ^ahrecheinli©hkeitsreohnungf!£  p6S 
(Ergebniss.e  der  Juathematik*  JQ  Springer^  Berll n 1933)  0. 

To  prove  theorem  ZQZ  we  put  sr/,/cv  j than  c^,  has  variance 

« 

1*  «*•  divide  the  interval  0 < t < % into  n equal  parts  and  put 


& 


f0*  l“lj2 


9 ® ® » 9 


n 


1 


6 


B? 


The  £*.  are  independently  distributed  and  eis=e,*60+  *s 

“in0®  ta5  li  fulfill  the  iindeberg  condition  we  sea  from  x®m&a  zt\ 

to8:-t  the  distribution  of  differs  arbitrarily  little  from  the  nor- 
mal  distribution  with  unit  varianoi®.  Heno®  e^«  & e'  i3  normally 
distributed  wife  variance  o^ai  ot 

w * 

• * "*  . ’ , r • 

-kQ  processes  defined  by  assu/nptioiia  Ip  2#  3,  &&&  4 will  be 
sailed  fundamental  random  prooesses  (abbreviated  F E p \ 

e b 0 ' & g -o 

'in  th®  following  we  shall  repeatedly  use  the  fact  that  the 
ul-isrioufciQii  of  the  limit  of  a sequence  of  random  variables  equals 

s.  **  *u  »*' •“«■»*»* 

wary  y.a.P.  is  Strongly  continuous. 

With0ilt  1088  of  Senerality  we  .ifcaU  a«a©»  o * ij  that  is, 

'*»♦< **  \ d9rlVi  n8it  aowral  lemas  needed  for  the 
prhcf  of  theorem  2.3. 

2|Jra«,  For  a > Q w©  have 

. m P 

fy.  K 

3 a IS  ) 


jV*;/2  dx  < .-*“/*/. 

a'  TO  ’ 


-=*roQf  :■ 


* ;• 

6 ,|»"A  /"' 


a 


a 


d:s  < jx©"2* 


dx  S © - / “k 


an  of  length  d and  for  S 

o*.  points  in  this  interval 


a 


p(?tt'S  > M>  < (8e*M“/8V4^F 


o8 


aM  the  points  t and  t'  to  S,  This  can  only  increase  the  value 
of  ^t/3s%S',fflU'3  c »^taout  loss  of  generality  we  may  further 

assume  x^—Q  9 since  we  could  otherwise  consider  the  process  — 

xs-xt  , Let  t0»t,  t1,  tg,  tn*  t#  be  the  subdivision  points 

of  S and  put  x.j  =xt  c Let  k%  {Uls2foco9n}  be  the  event 

{x^  < Mj*  COCf>xic3,j  < MgX^.  > M)  * The  events  are  exclusive  and  ex- 
haust all  oases  for  which  > M f 


«Ve  have  with  iQs  0 0 M > 0 


i>iAi#)X2a  > Mj^P(Ai)  PUn  > M|At)srP(A1 ) > M-^|Ai5 

> P( A^ ) P(x£-x^  > 0 |x,  «.  M ? 0 0 0()xH  «,  Mext  > M|  g 

our 

on  account  a£^  assumptions  we  know  that  x^-x^  is  independently  dis- 
tributed of  i1Coool)X|  and  has  a normal  distribution  with  E#ro  mean 

and  variance  fc  »fc.  so  that 

n x 


CQ 


i>ui»xn  i M>  t *lv  j ®SE 


-■  (x-x, 

II 


0 


a it  -T. 

n i 


■] 


d{x 


n i 
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1 r ( Ai  ji  ^ 
8 


The  event©  {A^Xg  > M}  comprise  all  case©  for  which  xa  ^ M and  ar® 
mutually  exclusive  Adding  C S0B ) ov@r  all  i we  therefor©  obtain 
{20S)  SPjxn  > M|  > P(Mtt,s  > Mj 


o 


She  left  aide  is  by  (2t6J smaller  than 


(201Q)  Z28 

T'ha  same  estira&t©  is  obtained,  also  for  < ~Mj  0 

Furthermore 

i’U'tVg  £ MJ  - P{elthaf  MU'S  * 1 °r  “tt'S 

g 

< swu*3  > |j  *P(«tt/s  < -|j  < iajT/^Tw)f&  /8S  9 


whioh  establishes  (207’)c 


Lemma  2Q40  Let  t and  consider  subdivisions  { tst^t^*  0 0 <i9 

t'  J such  that  t^-t^^^=  £/n  » 6n  c Then  there  exists  for  tv^rj 


positive  e*^  an  H such  that  for  an  arbitrary 


V rr 


8 of  points 


3 011 ) 


P(V 


% 1-1*1* 


< £$ i~l 


*•$  o o o £ 


nj  > 1-ij  for  a > 2? 


To  prove  (2011)  we  add  the  points  of  to  3»  This  will  at 

meet  decrease  the  probability  in  ( 2011 > ^ Since  the  distribution 

of  V*  ^ a ia  independent  of  the  distribution  of  that  of 
*i- 


g.  for  j we  have  by  lemma  203 
J **  J ^ 


U012J 
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where  k and  k*  are^ constants  Independent  of  ne  Sine©  it  is 

~nkf  n 

easily  seen  that  lim  (1-ke  ) s 1»  lesnna  204  follows* 

n^oo 

Lenma  2C5.  and  m^9  exist  for  every  internal  [t9t*j  « 


Consider  a sequence  of  subdivisions^^}  of  the  internal 
[t9t5]  of  aodul«  { 8 } with  lim  5 = 0«  If  S.  and  sa  have  both 

sufficiently  small  molul^  then  in  every  interval  of  length 

8 Q of  lemma  2®4  there  will  be  at  least  on©  point  of  and  one  of 

3 . • Hence  applying  lemma  2C4  to  the  union  of  S and  S we  obtain 

« 1 ,1 


Thus 


pMmu' s^^t' s 


I > 6)  4 J,  , 

V 

exists  for  every  sequence 


{Sn}  whose 


moduli  converge  to  zero  and  is  the  same  for  ©very  such  s©qu®no@0 

In  a similar  manner  it  is  possible  to  prove  that  plim  ie. 

1-^cpo  ” si 

exists 0 


To  prove  theorem  203  let  S ba  any  subdivision  of  modulus 
< 6/E  and  consider 

(20l3)  t S eg  D 0®nj  c 

*i~l  i 


tie  form  a new  subdivision  with  6/E  < < 6 by  deleting  points 

of  Sc  The  probability  (2cl3)  for  this  new  subdivision  is  smaller 
than  that  for  S0 


- 


. 

* 


- 


/ 

. 

. 

■ 
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By  lemma  w©  have*,  alnoe  the  distribution  of  V,  , . converges 

ci-lci& 


to  the  distribution  of 


0 


C 3 0 14  j P| ^ ^ ^ i«X*  > [1”8  exp(  -v&"/8  5*  ] 


Ei/5 


HI 


cT  /c?ir 


The  limit  on  the  right  of  (20X4)  is  1 for  6-^0  , which  proves  theorem 


p '* 

*'»0 


Theorem  2„4„  For  the  FCR0P-C 

(2,15)  P(Mat)-xa  > M)=  2P(x-0~xa  > M) 

we  consider  the  proof  of  lemma  203  and  write  t s b p t s a 

we  see  from  (2„9)  that  2P(xb  > M)  > P(M8feS__  > M)  c Here  S*  may 

i 

bs  any'  set  of  points  in  the  Interval  [&9b]c,  Lot  S|  be  an  ©lament 
of  a sequence  of  subdivisions  {S*}  whose  moduli  go  to  A®<~ 

cording  to  lemma  20s  plia  M - g M - exist bp  henoe 


(2„I6J  2P(rb  > H)  > P(Ma%  > U) 

W©  consider  again  the  events  A^  of  lemma  203  for  a subdivision 

of  [afb}  into  equal  parts  0 We  then  have  far  ; ^ ao,  with  a > li 

cs 

and  with  x&-  0 


42 

(2017)  JPUj,»X0  > M)  = > 0)  vP(A4,M  < xg  < Xj) 

= t ) ♦HAj*M  < X0  < X| } 

< Aj  I 4Pi  Aj,X£_^  < Xg  < X|  J c 

Let  e be  an  arbitrary  positive  number#  then 

U0A3j  p^i»xi-i  < x0  < xi)sP{H*xrxi-i  i e»xi-i  < x0  < xi) 

♦PUi**rxi-i  > e»xi-i  < xo  < *i> 

< J’UpQ  < xt-xc.  < ej  ♦Pjxj-x^  > ej 

(JS 

< P(AjL^£b-xct  < e)  ♦P(xi-xi-]L>  S) 

since  X|-xQ  has  larger  variance  than  Xj«xQ  and  both  ar®  normally 

distributed  with  mean  zero*  Adding  the  inequalities  (2017)  and  using 
i & 0 18 j w®  obtain 

C2J19)  P{xa  > MaMat  > M)  < |PlMab  > M)  <-P«0  < Xb-XQ  < t) 

♦ZPlx^-x^  > e)  0 

From  C20?}  w®  see  easily  that  ZPCx^-x^^  > eonvsrges  to  zero 

« 

for  ©very  positive  s and  every  sequence  {8,rJ  whose  module  converges 
to  z® ro0  Since  e was  arbitrary  we  have 

C 2o80|  P(xc  > Mstlab  > U}  <.  |p(Mab  > MJ 


o 
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Since  c maybe  chosen  arbitrarily  close  to  it  follows  from  (2o20 
that 

(2,21)  2P(xb  > MJ  < P{Mab  > M)  . 

The  inequalities  {2„21)  and  (2,16)  together  imply  theorem  2„40 

Corollary  to  theorem  2„4.„  Lot  S,p  3£0  0 „ 0 b®  a sequence  of  subdivi- 
sions of  the  interval  (tgt+x)  with  modulw  converging  to  zero  and 
consider  for  each  Sfi;  {t®t, s the  probability 

Pn*P^xt“xt  - °»*’«»xt“  xt  - 


then  X&m  P sO  0 
n-froo  a 


Yroc^%  a 


9 nave 


nlim  P„  - P(Mt>t+T-xt=  0)sH!Kt(ttfxt  > 0)=  > 0,-  0 

'Theorem  2.5,  If  />  > 1/2  » plim  (x.  -ij//  doss  net  exist. 

--  ' * ~~  — > o **  % * 

It  is  aaro  if  p < 1/2  t 

Th®  proof  of  this  theorem  is  left  to  the  reader© 

Pet  [ Is j be  any  interval  and  arbitrary  positive 
numbers  and  p < 1/2  C)  Than  there  exists  a 6 such  that  for  ©very 

subdivision  S * {a^stQ#  t„  9 0 0 0,  t^b}  of  modulus  not  exceeding  5 


nvt  iJlVVi/  < > 1“J} 
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Far  tbs  proof  we  need  the  following 

Lemma  _2_  ^6.  For  > 0,  6^  > Q*  k > Qa  k*  > 0*  * > 0*  and  5j+5g 

sufficiently  email 

C3oaaj  [1-k  expl-lsf  /c%)][l-k  expt-k'/dg)  3 > 1-k  exp[-k//{  8j*52)v  ] 
Proof  of  lemma  2 a 6s  The  left  side  of  (2022j  Is  not  smaller  than 

l-k{exp(-k'/8*)  ♦ «xp(-k#/*»p)  . 


Hero®  lZeZZ)  is  proved  if  we  prove  for  sufficiently  small  and 


FfSp  5g}s  «xp[-kV(5i^5g)v] -expf-k'/fij]- axpl-k'/op  > 0 

tfo  have 

lim  F(  5*  9 $2)  s 0 and 

Oyfr  Q X 

a 


exp[-k'/op 

*b  | 

i 

J>V+1 

81 

r8xp[-kV^5i,«'52)v  . 

expf-k^/Sg] 

*V  ) 

1 - „ * v*l 

• C j>j  + So  ) 

°2 

} 

} 
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The  function  x * ^ exp  {®k'/xv)  is  mono  topically  increasing  for 
sufficiently  small  x 0 We  ha  ye  therefore  aF/a^  > 0 0 

aF/a5g  > 0 for  sufficiently  small  5*  + dg  , Hence 
is  positive  for  sufficiently  small  5, * 6r>  t 

Ws  proceed  to  prove  theorem  2060  we  have  by  (207)  with 

l®£p  s v o S 

- €]  - 1=  «*p  ^a/as.Ji 

> 1 - k 9xp(-  ]£  /6 } 

where  k and  k;  are  independent  of  the  subdivision.. 

Thus 


How  let  S have  modulus  J then  by  lemma  2C6  we  may  combine  the 

intervals  to  the  right  of  (2023)  in  such  a way  that  all  intervals 
are  at  least  of  length  J|  and  at  most  of  length  6 c Hence 

Uc24)  B > jY-  k azp  («k//5T)J2^"“^8) 

and  the  right  hand  side  of  (2024j  is  arbitrarily  close  to  one  if 
5 is  sufficiently  small 0 ‘This  completes  the  proof  of  theorexn  2060 


[ 


1 - k expC 


h 


U.23 1 P 


- Pj 


,p 


<■  e8i-it 


o e o £> 


i i~l 


i i*n 

» >TT 

J i*i 
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A process  is  called  Gaussian  if  the  Joint  distribution  of 

Xj,  f x.  fooo0  X.  is  normal  for  ©very  choice  of  t^  t£0ooo0  tft  c 
61  *£  bn 

vV 0 now  consider  the  integral  of  a F0R*?0  and  pr&ve 
Theorem  20?r,  Let  x^  be  a F0R0PC.  with  variance  ot  0 The  process 

cic  /6or  i Con  C 

j[^=f  |x  di?  Is  a Gaussian  process  with  mean  zero  and  oo»«9totlon 

o 

fuuotioa  - £>  max(  t0 1*  j[min(tc  t*  }]*  - Slminttyt"  )]®*. 

The  integral  X^  exists  loi0m<)  by  theorem  1050  By  lemma  104 
we  have  S(X ^)s  0 and  by  theorem  105  for  t^>  t 


t t t t t y 

o J Jraint t0t' |d^dT^=o  Jld's-jV  j\  j\sV 

Q 0 ® Q 0 %* 


t't2 


Each  of  the  approximating  Hiemann  sums  is  normally  distributed  and 
that  itself  is  Gaussian  follows  from  the  following  lemma® 


Lemma  2e7c  Let  xn^ (x^0x|0oodrxj)  be  a sequence  of  normally  distrX 
touted  vectors  with  mean  0 and  assume  that  alfn  oxixJ  ™ @13  ex^sts0 

If  plim  x.  s x than  x is  normally  distributed  with  mean  zero  and 
n 


covariance  matrix  g 


11 


o 
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Proofs  inequality  (105J  may  be  derived  also  for  vectors  if  we 

interpret  x <*  a to  mean  that  the  vector  a-x  has  non« negative  com- 
ponents0  Lemma  Ec^  then  follows  easily  from  the  fact  that  for 

arbitrarily  small®  and  sufficiently  large  a 

• FnSa+5|  ♦ 8 £ > Fn(a-8}  - 8 „ 


where  f (a)  P Flaj.  are  the  distribution  functions  of  xft  and  x respeo 


3.  Friction  a!  Tk  e Orns'te,'*  -U/fUnbcc  l ,^>gef£  , 

We  have  so  far  in  the  Brownian  motion  neglected  the  effect  of 

the  motion  of  the  particle  itself  on  0 If  the  particle  has  the 
momentum  x^  then  the  random  impulses  will  have  a mean  value  proper?* 
tional  to  x^  it  Self  0 This  leads  to  the  equation 

!Xt+T=  Vt  +*t,T 


(2*25)  1 


fto*  at  * \ <lfw 


t > 0 f,  E(xoJs:  0 


wh»ro  again  s.  • la  normally  distribute  with  mean  Tain©  zero  and 
>0  * 

variance*  or^  and  is  independent  of  and  of  if  the  intervals 

{t%t'+x*)  do  not  overlapo  we  shall  further  assume  that 

xA  Is  normally  distributed  and  that  a is  a measurable  function  of 
* x 

T C 


I 


I 

• ' ''II 

/ « , " - * 9" 

* \ 


.. 


' 


■ 

. 

. 


* 


• ' - . - " 
i '”‘w  p . I 


. 


A 


- 


' • * X • 

- 


48 


It  follows  from  (2025l  that 


(2*26)  J 


" XUx^  aTsSXt  * %‘W 


E(x^  ^ |x.  )sa  x^sa^  & x* 

Ut1+t2  t t1^t2  * Ti  TS  15 


Honoo  ^ s a^  from  which  it  follows  that  a^s  $ 


ax 


and  since  a < 1 p a = e~^T  , B > 

% T 


p > 0 c wo  further  have  from  (-3025} 


and  ('2C26) 


a_  e*  _ + s. 


T2***T1  ""  **♦**!/ *2®  fet.rT^Tg 


Thus 


a 


! a?  ♦ a?  s a? 


n 2 _2  ^ 2 

a o $ q 


t2  t1  t2  Vt8  t1  TS 


or 


«°V°v 


(1-a^j^l-al  js  (l-e’<’^Tl)^il-*“to^Ta)  ,e 


Therefore 

(7,  37)  a®*®^X-®"2PT)  r p > 0 , 

Moreover  from  { 2025J  &n&  our  assumption  about  ^ wo  have 
jJ2028}  cr^  s a^a|  * crs( l-o“sPT} 

^ If  x approaches  infinity  than  of  approaches  0 That  is 

xUx 

to  say*  if  the  particle  has  been  subjected  to  these  random  impacts 
for  a long  time  then  the  distribution  of  its  momentum  approaches  a 
8 toady  state 0 


* ; 


. 


* 


■ 


. 


. 


. 


' 


■ * 


. 
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We  shall  therefore  assume  that  the  process  is  stationary*)  that 
Is  to  say0  that  the  joint  distribution  of  x,  0ooo0  xt  is  the  same 

as  that  of  xt  jv.hf)  0 00&  3 Under  this  assumption  of  s 

1 V x% 

and  it  follows  from  lz027)  that 

(2C29  J o'  y » a o2s  a2  exp(-pv)  for  t > 0 f; 
t t*ir  T 

/ 

The  x^  process c satisfying  the  assumptions  listed  above*  was  first 

considered  by  L0  Sc  Ornstoin  and  G0  Ec  Uhlenbeok*  As  will  call  it 
the  Ornstein-Uhlenbeok  process  (abbreviated  0o  yt{  P0)p 

We  next  consider  the  process  given  by  £2e53)c 


£So30» 


I 


1 

€S» 

V*8  ^og  t)/S|3 


! *«.  - 0 for  t < 0 

? CSSS* 

sue  have  for  > t 

E(vtvy  }^oh 

Jvg3=  0 


(2c  33.)  ' 


Moreover  the  ft,,0(,e  vt 

1 n 


the  process  is  a F0R0P0 


for  t > 0 


for  0<a<t<&'  c 

are  jointly  normally  distributed.,  Thus 

From  the  properties  of  the  y*.  process 


we  obtain 


\ 


. 

* 


' 


' 

- 


1 


. • ■ 

. ' r , 

■ 


' 

i 


' 
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Theorem  sft8a  Let  be  an  0oUoPo  Then 
(X)  la  continuous^ 

C XX  j x.?.  is  not  different  tables 

(XXX>  Mftb  and  exist  for  the  x^  process  and  it  is  strongly 

continuous^ 

(XYJ  the  following  equation  — > derived  from  theorem  $ ^ 
.holds  m > o 


c*o 


( B032)  P(e?\T  < M} ss  1-  2 JU//2j)exp(-xB/H)  dx 

«oo<^<(l/2p)log  t „JjL 
■ ' <r/i" 


In  equation  (2032)  we  have  written 

P (y.  < for  P(M  . < M) 
&<t<b  *»  w 1 **  - s 


where  1®  the  maximum  of  the  process  in  a < t < b c.  This 

notation  will  also  bo  used  In  what  follows,, 

i 

Equation  does  not  seem  of  great  use  as  it  stand s0  but 

we  can  obtain  from  It  a bound  for  P (x  < Mj  as  follow® 

P (aFx  « M)  <.  P («PV.  < M)  < P Cx_  < ***P*lj  • 

t,  --aa  '"S  l==5'  " ®se  „ esmt  * «&>  <=>  ^ 


- @i>  <?<? 


■*lSTS*s 


and  th m 


5l 


(2c33j! 


/*;  * /o 

P (xT  < M)  > 1-  2 f 4=  e”  x /d 


Tl  <•.%<*%  o 

Af00  c=>  #5 


dr 


•ft-* 


t 


Theorem  a090  Let  x^  be  an  0olToPo  The  integral  I^s  jx(it  of  this 

© 

process  exists  then  X0!0m0  end  fear  tg  > t?v  its  ©©variance  function 
ia  given  by 


(8,343  R*  ...  - SC 


2 r ~02  -,k'  -o  -i 

B ,.  = mrnm  I 9 *♦«  <>2pt,”l-*‘  “ 

*1”S  o2  1 A 


9 


( 80  35  3 

is  a F0R0Ptt 

Proa £:  For  t„  > t. 

' 2 ™r*  j 


t©  „ „ , 

a a r * ® r ® 


®t**  P*W  * XfX0 


have  by  theorem  105 


s 0 


t 

2f*f 
%«,  <!» 

© © 


dTlfi,r2 


<$»  l$* 

15  -fa  **  <n>  0 f \ ,n  ca>  ‘#“5  jj 

r @ 2 1 


•5*0  , 

0-  T 


dT24,fl 


1 


2 r 


'X 


<rse 

2 


J 
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anfl.  for 


EUt-X0)2=  |sp«'P*+pt--l)  , 


A straightforward,  calculation  gives  for  tg  > tj  > > 8.  g 

C 2C  36 } 1[{X  -X  ~x  >3 

"2  *1  °2  *1 

ps  pa  -ptj  -pt 
o -e  ue  ”-e  A\ 


-%  2[(xt-zt  jU B~x  )] 
Px  "2  l 3 8i 


Uein?  FU/.J1  “ s(x  i -.U«o  X8*fe  ~S  ) 


f'a 


* h-?»o  "•““-g 


we  obtain  from  lemma  107  and  (2„34j 


2 

0 


(2„a?i  s«xtxgj 

From  (g03fj  we 

f?.0s8) 


P 


[a if  t > 


E[(Xt-Xt  )Cx  -* 

2 “'l  2 1 


if  * < 


0 


* ~ ~ ( 0 


~ - S£llg^-Xg^|  (Xj-X^ 
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It  Is  easily  sear,  from  th®^Jgc.3SJ  and  (2028)  that 


(a035) 


B 


t P<W  ~ XQ 


has  th«  property  that  fos?  tg  > tj_  > s,  > the  difference 


Ba.  ®°  B 


^ is  independent  of  Bg  - BQ  - aM  sine©  is  normally  die 

tributed  it  is  a F0R0P0 

From  iz0 35}  w@  have 

Ihu@  for  ©very  function  fit)  for  which  the  operations  indicated 
b@l®e  have  mis&aing^w©  hav© 

t t t 

S_,  .. 


'fCtj  tet  --fsjfU)^  at  ♦ Jfltj  fiBt 
a a 


W©  may  writ©  ^B0S^|  as  a stochastic  differential  equation 
Cg04Q|  s - px^  &t  ❖ e . 

In  the  form  Cso'40j  a stochastic  differential  equation  too  moaning 
OT©a  if  th©  proces a©s  are  not  different iable0  This  interpretation 

i 1 . ■ j ■ ■ 

of  a stochastic  differential  equation  is  du©  to  J0L0Boob0  Th©  ©qua- 
tion  Cso40|  may  to  interpreted  as  th@  Ion  of  the  motion  of  a 

partial©  (x^  boing  its  velocity  at  time  t$  subject  to  fandom  impacts 

when  the  frictional  for©®  ie  proportional  to  its  velo@ity0  On®  ©ould 
also  interpret  x^  as  an  electric  potential  subject  to  random  changes 
when  the  d©or@as©  in  potential  is  proportional  to  the  potent ini  Itself 
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{Thus  we  may  consider  a oqjidenser  which  is  charged  by  a randomly 
fluctuating  current  and  at  the  same  time  grounded  through  a resist 
anoo 1 0 In  short  0 ©action  (ao40J  describes  any  situation  in  which  a 
quantity  is  subject  to  random  changes  and  to  a systematic  decrease 

proportional  to  x^  iteelfc 
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CHAPTER  S 

ESTIMATION  OF  PARAMETERS 


In  th©  preceding  chapter  we  discussed  Markoff  processes ^ we 
shall  now  apply  our  results  to  obtain  estimates  of  the  parameters 
determining  these  processes  from  observations.  In  our  estimating 
procedures  we  shall  assume  that  we  have  at  least  one  curve  at  our 
disposal  registering  the  values  x^  for  all  values  0 < t < T 
Actually  it  would  be  sufficient  to  know  for  any  dense  get  in 
this  interval^  This  procedure  may  not  seem  realistic  sine©  we  never 
observe  the  process  for  every  time  point  o Every  method  of  regis- 
tering the  curve  described  by  x^  will  itself  affect  and  In  part- 
icular smooth  the  path  curve  of  Thus  what  we  observe  is  really 

a modified  process a 

However  the  methods  of  observation  may  be  so  refined  as  to  give 
us  the  value  of  x^  in  a large  number  of  points  and  at  any  rate  the 
variances  of  our  estimates  if  obtained  from  discrete  points  may  alf>< 
be  computed 


The  FoR0Po  is  completely  known  if  the 


tejR. 
the  constant 


Eix 


is  known 0 We  first  discuss  the  estimation  of  the  parameter  o of 
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Theorem  3,1  If  x*.  la  a F.RdP0  and  if  it  Is  known  in  a dense  set 
in  an  arbitrary  small  interval 9 then  it  is  possible  to  estimate 
the  parameter  o with  arbitrarily  high  preoieion. 


ftoofi  Assume  that  H observations  are  taken  in  the  interval  0€ 

let  t g T/21  and  x in  6 Q,X8E9 , , * ^ the  sample  value  at  the 

time  nx.  Since  x ia  a FeReP0  the  variates 

V 


x(n.l)<e  ^n-ly^T 


a 


/r 


/x 


8 2s  8 ( ® «Hj 


are  normally  and  independently  distributed  with  mean  %m*o  and  varl 
o0  The  maximum  likelihood  estimate  of  the  variance  of  Jn  Is  there 
fore  given  by 

H • N 

i30l)  °8f  * J^l 'Xbv~  x(n~l)V  j^$/xnv~  x(n*lJv^ 


v/e  have  Ease,  Moreover  5$  /o  has  the  ohi»aqu&re  distribution  wi  1 
K degrees  of  freedom,,  Its  variance  is  therefore  gKe  Henoe  ^ has 
the  variance  Bo* fB  and  this  ©an  bemad©  arbitrarily  email  by  UMi. 
I large  enough 0 

Thug  if  it  were  possible  to  observe  the  procesa  completely  in 
any  interval^  however  small,  we  oould  determine  o accurately. 
Actually  however  every  registering  instrument  will  introduce  a ti 
lag  and  will  thus  smooth  the  process,.  We  may  Infer  however  from 

our  result  that  the  points  for  which  we  read  the  value  of  x shoul 

t 

be  spaced  as  closely  together  as  possible*,  That  is  to  say*  as  ole 

as  ie  consistent  with  the  assumption  that  the  values  of  x ~x 

■t*“f  t 

obtained  still  represent  the  actual  values  supplied  by  the  £ , R 0 !* 


5^ 


More  generally  we  prove 
Theorem  3e2 


If  y , is  a stochastic  process  such  that  y 

t *5 


Z.  * f { 

t 


where  x.  is  a F,R0P0  and  f{t)  a function  of  bounded  variation 
t 

satisfying  in  (QST)  a Lipsohitz  condition  < M for 

some  M^then  it  is  possible  to  estimate  Vne  parameter  o of  the 


x^  with  arbitrarily  high  precision 


F.R0P 

Fro*£:  Let  again  t s T/H  and  consider  the  sample  points  x^{m 

• H , . . \Z 

28..„eH).  Denote  by  o « ^ J_  yRT  °yUr4).T.  ... 

® n«l  x 


9 I $ 


Then 


%>  “ pj 


tx,jT=  ♦ f(nt)  = f(n!It  )] 


8 .2 


CT 


Her©  and  in  the  immediately  following  formulae  the  summation  is  to 

be  extended  from  n-1  to  n*  B0  We  write  also  n^l  for  (n®l)  to 

simplify  such  expressions  as  x,  „ x s , y,  ‘ s or 

(n-l)T  n4t  (n-lR  n~X<¥ 

f[(n«iW]  s f (nTix}  0 Then 


♦ -ft  Z[«lax)  - KEltJ 

♦ -i-  2[f(n^)=  f{n=l  *?}  l2 }2 

CT 


We  expand  the  right  member  of  this  expression;  a considerable 
simplification  follows  from  the  assumption  that  x is  a F.R.P,,  we 

w 

use  in  particular  the  fact  that  5P£~2ELs  is  normally  distribut 

/or 
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X 

with  zero  mean  and  unit  variance  and  is  independent  of  ,mT i;...,JmL£ 


0 % 


for  n0  Thus  we  obtain 


El 


|,2„i  + 2 + 2r  [f  (n-t)  - )]3  4 r [f(cT)  - f(n^Ic)  3 2 

o'  " i • 1 * 1 t-  c X i»  “ O X 


jg  { t [f(nT)0” f -5-1  y 


and 


Etl)  - i»i.T [f(nT) 

o c<? 


Hence 


E(c  - o) 2 - -S22  4.  is  y [f(nx)  - fin^T)  I2  * _L  (y  [f(nx)  - fU-1  -elf  )1 
H H2  T jjS-  It,  . J 


Thus  o converges  in  the  mean  to  c and  o»c  is  stochastically  of 

the  order  1 fj§  0 In  fact  E(c-c)2  < JtSU  + iS  MV  + i(M¥) 2 

y ~ E Jr  N*g 


Here 


f{t»T)  - fit) 


< M and  ¥ is  the  variation  of  f(t)  so  that 


also  ¥ < MT  where  T is  the  length  of  the  interval. 

Thus  in  estimating  the  function  f{t)  w©  may  assume  o to  be 
known*,  if  we  know  y^  in  any  interval  completely^ 

tf©  shall  discuss  two  examples  of  the  function  f(t}9  In  the 
first  we  assume  that 


fit)  ^ at 


(Since  we  can  always  consider  the  process  this  assumption 

is  identical  with  the  assumption  f(t)  g at  * b e ) We  then  kno7 
that  the  y\  are  normally  distributed,  and  independent  in  non- 

over  lapping  intervals  with  mean  at  and  variance  o%  0 Hence  the 
maximum  likelihood  estimate  of  given  the  values  at  time 
09  %9  coo*  becomes 


a 


1 

1 


z(ya**  - y 


n®1!  v 


) 


v-s  - y 


Q 


Its  variance  i® 


?or  the  second  example  we  assume  that  fjt)  is  given  by 

m 

fit)  sJZ  Cameos  Jt  ♦ p4sin  ^t) 

J«1  J * 

and  that  the  values  of  y are  toomx  in  ths  Interval  (0,  Elf)  ani 
that  y„  * Q„  If  we  4ust  choose  the  values  yQ»  y^»  y^. , yJt 

where  nv  e g 7J"»  the  maximum  likelihood  estimates  of  the  and  £?. 
will  be  given  by  those  values,  which  minimize  the  expression 

& m 

ZL  {yiT  - G.[003  i^v  - eosti-l)  Jv] 

i-1  4 1 AT  j»l  * 

-r  C fain  ijv  - sin{. i-1) 

i-l  i 
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Henoa  the  maximum  likelihood  equations  are 

n I v/,.a  . 

(3«20l)  21  (ylT-  )[00S  ik?  « ©os(i-l)k?] 

n m A 

s 2 I ou[©03  i^T“  coe(i-l)  Jt][oos  ik.T~  oosti-Dkr] 

1*1  5*1  « 

a m A 

^ F ]T  s«[sin  I^t -sinti*-!)  Jt][oos  ik^  - oos(  l-Uk^] 

l»l  3-1  * 


k «•  1 p o c © 9 ^ 


and 


n 


(3„2o2)  E ty{r-  yp-j.j  ) tain  ik-r  - 8 in!  i-‘l)k<S 


n m 


= L L 

ui 


JT  d [ooa  y-s  ~ oos(i-l)  JxHsin  ik-r  - aia{ 


3 


a m A 

t r.  z p-csm  y* 

i«l  4*1  J 


- aln(i-l)  Jt][ela  ik-r  - 8inU-l)kr] 


k s£  1,  3# 


e a o # 


m 


If  w©  dirid©  the  first  of  these  equatione  by  t and  let  % go  to 
setro*  we  obtain  because  of  the  orthogonality  of  .the  sine  and  cceinfe 
functions 

zr 


■k  | sla  kt  dy.  ss  cl*  l sin  *kt  dt  ^ cl  irk 

vrt  & k fcV  •£ 
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The  rules  of  calculation  for  the  integral  in  the  left  member 
of  this  equation  are  completely  analogous  to  those  for  the  ordinary 
Riemann-Stieltjes  integral.  Integration  by  parte  on  the  left 
gives  therefore 

j y oo s kt  dt  » a^Tf 


and  thus 
(3.3,1) 


A 


.STT 


sir  J H 


cos  kt  dt  s . 


1 

71 


! 


zr 

sin  kt  dy^ 

0 


Similarly*  we  obtain  from  (3.B.2) 


Err 


2 - 


k j’  oo  s kt  dy  2 f k*  p 

J0  * 


Integration  by  parts  gives  again 


(3.3.2) 


a 


+ V 8ia  ^ ^ * A C'C06  kt  d7t 


The  integrals  in  (3,3.1)  and  &303,2)  are  to  be  understood  as 
stochastic  limits  of  Riemann  sums*  and  it  is  easy  to  see  from  the 
corollary  to  lemma  1.7  that  these  Riemann  sums  converge  in  the  mean. 
From  lemma  1.4  we  see  therefore 

E(\)  * C*  , E(gk)  s ^ 

and  similarly  from  lemma  107 
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o*«  sfct  n®  1 4 8in  kti  6111  ktj  0 {(y*  - y*  Hy+  - yt  )} 

afc  T ^ 5^0  1J  1 3 \ ti.i  “4  S.i 

5f*0 


nJaNr  o.  - ra&xj  t^  * t„  | and  5,  s maxj  t . - 1 4 | . 

l i sj  2 j'-l 


Since  the  increments  of  y in  non- overlapping  intervals  are 

w 

independent  of  each  other  we  havs 


so  that 

( 3*85 

% 

and  similarly 

(^»9) 

°1  s ip 

Further 

(3.10) 

ft  s ap  1 
Tc  at  Te  .£, 

Ca  * s 0 

“k*3* 

) singlet. 


c 


£ir  9 

f sin^kt  dt 
J0 


= 0 for  k Z 


Frc&  lemma  £06  we  see  moreover  that 


are  normally  distributed* 


Suppose  now  that  we  take  observations  in  the  Interval  {0* 

so  that  we  have 


*z 

o. 


ii) 


f(  t) 


7 


[a^  cos  21m  ^ sin  r2im-|] 


r> 

wo  put  .an  yj  - yt  . 


<*"T> « ^ « 

xhofi 


I & : 

- * 


o 


2 


0k 

»pv»®  maximum  likelihood  estimate  a and  B?.  then  oeooaie 


iw 


A 


IS 


1 ■ ^ 

fE 


t A 


o,_  88  - •sag,  J sin(27f& 


0 


rp 


A 

Pk 


~ 4- 

| eo-a  (2rk 


o 


2 

0-^ 


& 


ID 


v *?S  _ *2  n <0 

Pfc  s-12  ** 


v ■ ; - 

i ors«sTe*iWrifi>^-5f^»"r«» 


a «s£  0*  ,CTa  a s&  €f>&  £ » t -'5- 


V 


a,  a. 

k t 


?y  P* 


u- 


a confidence  region  for  the  avp  83p  M # 


where  the  sum  runs  over  those  terms  (a^~  a^)  9 ( p ) which  are 
not  zero  by  assumption,  and  $,* 2  has  the  % 2-distr  ibution  with  the 
number  of  degrees  of  freedom  ec^ual  to  the  number  of  terms  in  the 
sums  on  the  right.  The  estimates  a^f  p^  are  consistent  in  the 
following  sense.  Suppose  f(t)  is  given  by  (3.11)  and  we  observe 
y^  in  the  interval  YT  where  V is  an  integer.  We  then  have 

^ 008  2jr"Tr^  * Bln  3 


so  that 


2 

Cf^ 

®k 


2 

a* 

Pfc 


Y To  s To 

2 rMk*  3 JT^Vk^ 


and  thus  pi  in  cL  = gl  . Similar  lyp  plim  h*  3 0 

Y-f>co  s * Wes  ' * k 

Q.4.P. 

We  now  turn  to  the  discussion  of  the  0„Uo£„  given  by  (2 
and  prove 

Theorem  3.3  If  x^  is  an  0oU9Po  determined  by  $ha  two  parameters  'ft  and 

2 

or  and  if  the  values  of  x.  are  known  in  a dense  set  in  any  interval 

% 

2 

0 < t < Te  then  it  is  possible  ,to  determine  c p with  arbitrarily 
high  precision. 

Proo^ : We  form  with  s T 


(3.12) 


J) 


i r vXnT  " ^rt”!  *r  ' 


we  have  E(D)  * JBg5(l-a J - 2o2  . 


For  t-I^O  this  converges  to  2a  p, 
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We  now  compute  the  variance  of  De  For  this  purpose  we  shall 
need  the  value  of  } with  t < t1  < t,f  < tHS  e 

We  have  for  t < V < t18  < t908  on  replacing  x^*,  hy 
o f zf  * et^tm-tn  &&&  analogous  transformations 


E(xt  xt,x|,,)  = xt,  ♦ et»jtn.t<)  J 


2 , \ / -»  2 » 4 
s atr,«t8  ^ * aV  ~t ^ ~ atw“t9  J ° 


and 


E(zt  x?.i  * BCxtlat*-*  xt * et,t»=t,s3 


= »t*-t  E(zt)  + »*•-*  E(x?)  <r‘i(1°  aiS»~t: 


s c^£3a^i_^  ♦ 3a^i«,$  •=  s Sa^»=^« 


Thus 


and 


E^xtxtsXtn)  ~ a !>t"~t9  at9“t  4 at®®t^"  atw®t3U 


4 2 

c ^at3-t  41  ^atw~t0  atf,-t^ 


4 g 

s ° ^atm-t"  at3=t  41  ^atm-t,f  atn-t9  at8~t' 


A 


s a at*»t  * Satm-tr'  atn=>t' 


or 


13)  E(xtxt?xt^tm) 


s a 


exp r=p(  tre  «tn*t8~ 1 )1  + Z exp£®p{tw 
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For  n < m we  find  easily 


^3 


xax"  xn=I  t ' 1iet°  zm=I  t ] 


E < (* xztt“xn=l  t } ^ £(a<t”  1 )x5=l  t + ES^'ri  t 3 *) 


2 ° 


1}  CE(xnx  J4=Ix1  “ j4=1t)  * ^*£=It*S=I  x ] 


♦ 2 a4  (!♦  a^)]* 


From  (30l3)  w©  838  then 

$H«-  xn^X  x ] 8(xbt  “ xi^I  t ] * J 


2 4 -3B(a~n-l)T  =8t  «BT[2(m-n) -1] 

T“l)  0^|U2e  1 - 2e  ’ -4© 


«2B(m®n)T 

♦ 1 ♦ 2e  + 2 t 2 a ! • 


Since  a & e PT  we  finally  have  for  m > n 


(3014)  #*  - Z 


<=>  JL — -=»  ; • X — -b* 

n«l  t mT  m~X 


n 


(l^a. 


2 "•28(m<:»2i'*l  5 x . 

) 9 ' 1 


i 


* • 


% 


V 


Farther 


E(X;are»  X£SI  T 


4 


h;  lift,  «=. 

IT 


+ €J  “*==3 

n^i-r  - nrai  *?: 


4 


2 2 _ 2 ^ 2§ 
l)4  ♦ 6(aT-  X)  Bixjq T )(1»  a“)(T  ♦ 3a  U~&T), 


3 a4  [l 


4 , Z 2 2,2 

1)  + 2(a  « X)  U ° <• 


3a 


4 


2 


that  Is 


L. ) ^ ♦ 2 ( X - a® ) ♦ S X + &r ) ] * 

J 


4 * 12  c4(l~  aT), 


SVom  (3,125,  (3,14),  and  (3„l5)  wa  have 


4 


j a _|_g  (1-  a^)  12  H ♦ 4H(H"1)  ♦ 4(1-  a,,) 


Ml  rri<=>n«a 


bo  that 

£ _ 
O’  « 

D 


4cr4d”>  ajtl_  fgH  + ( 1 » aT ) 2 


►2p(m®n®X)T 


1 

J f 


now  compute 


I 7 <*'>“■ 

$38  MzffcH 


Y_  (»v”n 


as 


By  using  the  formula  for  the  sun  of  a geometric  series 
it  is  easily  seen  that 


•I  ^4 

Lm~n 
b 


3 1 - b 


H 


■*°  “ ® C 1 88  b ) 


hence 


A e 


3 


1 • &> 


23 


i-a2  (1  = a2)2 


We  substitute  this  in  the  expression  for  cr  and  obtain 


B 


4 cf  (l~aT)^  f E(l«a,r) 

3*t  L 


1-a* 


1*sl  (l*aJ2  j 


or 


f30l6) 


£ 


4c4(l-ax)2  r H-l 

u 


I 


2 2H 

<?  “i 


(Lap 


2 


Equation  (50lS)  shows  that  a"  can  he  made  arbitrarily  small  by 
making  H large  enough,  In  fact 


lim  qu  s 8 B^  and  lim  E{J>)  « 2 8 / 

=&rn  WTtr  T)  r ' V 


3-^qo  i 3 D 


2T-£ao 


£(D-2Pa2)2  = a2  ♦ 4a4(I~£!2 


\6 

>) 


also 


and 


Kjysu-apoSg  S2//ei  [iisl!!  = p ] 

and  therefore  since  tsT/K 

lim  Ef/^D  ~ 2p03Q”  0 
Tf  “*C® 

£ 

proceed  to  prove  that  the  limit  distribution  of  </$( D - 2($cT^ 
is  normal.  This  may  be  seen  as  follows:; 


nZi  x V5  fe 


H~'  [(eT-^)x 


■»  g 1 

' nT^TJ 


m~  ' ht  ?,  t< 

SBSS^SKt  A 


or 


, t „ n_  i ^ 2 2 (*T-1)  y _ 1 y gnv. -;. 

(3.17)  */®  % L,  7'a%  * r L '■ni'n'r.T  ' ,/W  I— 


ti-l  J° 

“nt~aT,t  ' 0 


The  last  sum  is  a sum  of  independently  distributed  variables 
all  with  the  same  distribution  and  converges  to  the  normal  dis- 
tribution by  lemma  20X0  Thus  the  normality  of  the  limit  dis® 
tribution  will  be  proved  if  we  can  prove  that  the  first  two  sms 
in  ( 3017)  converge  to  zero  & 
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tfe  therefore  put 


(3018)  ~ 2 x^  and  %Q  & -JL  f x £v_ 

1 ^ * V nT  Z % o nT  nT*  ^ 


»Ys  ha~e  by  (3cl3) 

4 


2(zf) 


5 


t* 


* U^1'~  [52  * 25  * 4??r*Pln-n)T]# 

3 JW 


The  double  sum  in  the  bracket  oan  be  easily  determined  and  we  have 


E(zi> 


^(a^-l)4  r 2(l*«w) 
5 <* 


¥ 


t 


1 -a' 


i 

H U=a2)2i. 


Clearly  lim  E{£,  ) ® 0 

5<“f>oo 


Fur  ther 


, a, 

E(zg) 


*«  a //=*  / 


i jsu2  *i^L  „*u-4) 


f-  nx  ^ *e 
o 


and  therefore  lim  E(20)  = 0 , 

5~#oo  2 


Therefore 


l0i*m.  2^  ~ l0i0aio  Z ^ 0 so  that 
w 4 5 oo  ^ 


CD 


h'-t 


5 oo 


d - •%  £ 


/H  Si  x 


n 


and  since  the  second  term  on  the  left  is  normally  distributed 
2 


with  mean  2 


it  follows  that 


2j3a^)  Is  in  the  limit 
4.2 


normally  distributed  with  mean  zero  and  variance  8cr  p 

2 

To  estimate  a separately  one  might  use  the  estimate 

*2  \ 2 . 

Q = -m  J ax‘ 

A o b 


Its  variance  is  given  by 


hT  T 


-h  I X E(xtV  )dt  d4’  * ° 

T2  o o 


4 


%)  f f e 


*'  ■**  ««•<*  . /;*  .-***'-»>« 


* O'  t 


2<r4  + g4(e"S3~-  1) 

pT 


A 

If  p.T  is  large  enough  compared  with  a"  this  comparatively  simple 
estimate  may  be  quite  satisfactory. 


CHAPTER  4 


THE  GENERAL  DIFFERENTIAL  PROCESS 


W@  stoll,  consider  processes  x ^ with  the  following  properties g 
Cl}  x,^  ie  a continuous  process  (not  necessarily 
strongly  continuous  )| 


2>  L8t 

„ X.  C , X 0 6**0 

6x°  i *2*  *a 


Th®  random  Y&riablea 


are  completely 


independent  of  each  other  if  the  intervals 
(t1Dt^i1}c(t2l.li2+-v£)F  ,0,(,(tnrtn4-TnJ  do  not 


OTorX&pg 

{3}  Th®  distribution  of  ^ is  independent  of  t c 

Prooeagas  satisfying  thss®  thro©  oonditions  will  b©  called 
differential  processes 0 In  this  chapter  we  shall  find  a general 
expression  giving  the  distributions  of  x*  _-ac*  for  all  possible 

rr  % « 


^<4/1 

^ differential  processes,,  Processes  satisfying  equation  (20X|  and 
assumptions  10  z0  30  of  chapter  2 are  differential  processes  of 
second  order  and  we  discussed  in  chapter  z the  special,  case  where 
ac*4.‘f~zfc  normally  distributed*, 


We  stoll  first  discuss  another  special  c&ee  in  mioh  the 
increments  *»x^  have  a discontinuous  distribution*  A prac- 

tical example  for  such  » process  ia?  for  instance©  the  total  of 
insur  fence  claims  raised  against  an  insurance  company  as  a result- 
of  randomly  distributed  accidents*  ah  important  special  oase0 
fundamental  also  for  the  understanding  of  the  more  general  problem^- 
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is  that  In  which  increases  a randomly  distributed  number  of 

times  within  @v#ry  time  interval  but  each  time  by  the  game  amount n 
ft’©  shall  call  such  an  increase  a 8hotc 

make  the  following  assume t longs 

(1}  The  probability  jP^th&t  k shots  will  occur 

during  the  Interval  (tpt^x|  is  independent 
of  t end  of  the  number  of  shots  that  have 
occurred  up  to  and  including  the  time  ta 
iz\  The  probability  q^2^  that  more  than  one  shot 

will  occur  in  a time  Interval  of  length  x 
is  of  smaller  or&sr  than  x0  la  symbols a 

qf,2^  s o(t)  or  limlq^/T)  = 0o 
% x o f 

pi^'  la  a measurable  function  of  xa 

m ^ 


vVe  clearly  have0  if  Xj*x9  — * 


1 4cl) 


4°» 

1 2 


From  { 4 0 1 j and  the  measurability  of  It  follows  that 


ax 


and  since  p^v*  <.  1 we  have  a < 0 0 Moreover c if  are  any  shots 

to  be  expected  we  must  haw  a < 0 9 ss»]i  where  pi  > 0o  Thus 


{0)  «.  e-l« 


o 11  > 0 


U*2) 


o 
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We  bow  divide  the  interval  {t*t*'r)  into  N parts.  Then 
for  sufficiently  large  N the  probability  that  two  shots  will 
occur  in  any  of  the  intervals  oan  be  made  arbitrarily  small  so 
that  If  denotes  the  probability  that  k shots  will  ooour 

during  the  time  interval  % - 

(4,3)  pvSJ=(k)  [l~  «cpl-££  {H-kj3  ♦ o(l)i 

For  14 ->-09  we  then  have 

(4 0*)  p(3t*-  dlfei?™ 

*t  k* 

The  distribution  (4C4)  is  the  Poisson  distribution*  its  mean  and 

variance  are  both  equal  to  p.v  0 

We  next  consider  a situation  in  which  the  assumptions  (1) 

(k) 

*2  if  3)  regarding  hold  but  where  the  increment  of  x+  at  each 

shot  varies  and  has  itself  a probability  distribution  fyix)  =s 
and  we  shall  also  assume  that  the  increases  in  different 

shots  are  independent  of  eaoh  other 0 If  $-(x)  is  th©  distribution 

of  the  ©u m of  k independent  random  variables  * eaoh  with  distribu 
tion  &(x)0  then  th©  distribution  of  the  total  increase  provided  that 
k shots  have  occurred  ia  $k(x)  c Thus  th©  distribution  of  xt 

is  given  by 
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(4,5} 


P(xUT_Xt  ^ - 


00 


Z 

k*Q 


with 


for  A < 0 
for  A > 0 


Is  the  following  we  shall  need  the  characteristic  function 

[abbreviated  ocf0 J 


U.&)  f^U)  = E{8Spfii8(xt^T-2t)3} 

cr.  th © distribution  (405jf:  An  er.»y  calculation  gives,  if  gCs| 

denotes  the  c0f0  of  ${Xi 

*-(•!«  £ - ^ lVR}*  Cgtsj]*8*  exp{jiT[gl»J  ““!]}• 

v ko  X t 

The  distribution  (405J  is  called  the  generalized  Poisson  distribu- 
tion. 


,tfe  return  now  to  the  general  differential  process.  ^e  hare 


at,T®  stci*  *t*jf  i *#i 

an  n a 


n 


Hence r lfdL(s)  denotes  the  o,fQ  of  6.  we  have 

A 16  - t ot 


f 4,8) 


#fCs?  ■=  t*f xCs>  31" . 


n 


Froa  the  continuity  of  x*  it  follows  that  lim  0_A<r  (s)  as  *3  is)  c 

* &£*£»<&  1 

T h' iB  (4^8)  ispli 6 8 

U.9?  0^5sj  = 10^(8 _ 


' 
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On  the  other  hand  every  family  of  distribution  functions  H^(x) 

whose  characteristic  functions  satisfy  aquation  (4„9)  is  the  dis- 
tribution function  of  the  increment  of  some  differential  process, 
Hence  the  general  form  of  a differential  process  will  he  found  if 
we  find  the  general  form  of  characteristic  functions  0(sj  that 
satisfy  the  condition  that  for  every  % > is  a o0fo  A 

distribution  law  whose  0ofo  satisfies  this  condition  is  called  an 
infinitely  divisible  law  (abbreviated*  !0d0l0j0 
Oar  main  result  will  be  the  following^ 

Theorem  Uet  ^f(sj  » log  0(e)  „ The  function  0{s$  is  0ofc 

of  an  infinitely  divisible  law  if  and  only  if 

■<$»  oo 

(4,10)  ty{«)  a iea  ♦ ? (e,i0X- 1-  4&(x; 

~Q©  1*XS  X^ 

where  a la  real  and  G{xj  non-decreasing  and  bounded  and  the  inte- 

2 

grand  is  defined  by  continuity  to  be-,®  for  x«G  0 

fundamental  for  the  proof  ©f  this  theorem  is  the  powerful 
continuity  theorem  of  P0  Levy0 

Continuity  Theorem 8 Let  {F^lxj } be  a sequence  of  dlatrlbution 
functions*  the  oorresDonding  sequence  of  o0f0580  The  se- 

quence {Fnix})  converges  to  a distribution  function  F(x)  if  and 
only  if  f Ce)  converges  to  a function  fls)  continuous  for  bs  0o 

U8J  Theorem  4C1  is  due  to  PD  Lovy  fsee*  for  instance*  his  "Theorie 
do  l' addition  des  variables  aleatoiresvv9  Cauthiers  Villars 
P&ris^XSS1^  p0  180)  0 The  following  elegant  proof  is  due  to 
M0  Loeve*  University  of  California  rubl0  in  3tato0  vol0  1* 
W0^053-88Cl950jo 


I 


\ < ■ • '■  < 1 ( ' * 

* ■!.  " ' ' 


* -■  • '< 

. 


I 


■ 

' 


/ 

, < • * ■ ' • • ‘ ■ * : J 


* 


■ 


Far  & proof  the  reader  is  referred  to  H0  Crambo  Mathematical 
Method©  of  Statiatloso  10 jQ9 

Via  shall  need  this  theorem  in  the  following  slightly  mor® 
gonsrale  fora0 

Corollary  to  tfog  Continuity  Theorems  Let  {F^lx|}  be  a sequene© 
of  bounded  monotone  functions*,  Fn(~oo  ) =s  0 and  let  {f a { s ) } be  the 
sequence  of  their  Fourier  transforms 

U«U»  t U)*  / elM  «F  U)  • 

-00 

The  sequence  {F  (x)}  converges  to  a bounded  monotonia  function 

Fix*  and  lim  [F  5- F (-o©  j]  « Floo  5 - F(-o»  ) if  and  only  if 

« » 

the  sequence  } oonvergee  to  a function  f(s}  continuous  at 

a ss  0 0 

The  corollary  follows  easily  from  the  continuity  theorem  in 
observing  that 

V°J*  X «.U,  - vn . 

Hence  the  variations  V„  of  F (x)  ©onverge  to  f(0jo  If  fCQj^Q 

a a 

then  «£-«—»  converges  by  the  continuity  theorem  to  a distribution 
T*a 

function  H(x)  aM  Fa(x)  0 on  verges  to  Fix)  HixjfCO)  0 If  f^OjssO 

then  lira  F_{x)  - 0 and  the  theorem  also  holds0 
n-M»  a 


■ 


- 

. 


. 


• > ■ I 

^ I 


. 


TO 


shall  also  need  the 

Let  {F  (xj)  be  a sequence  of  distribution 

AA 

functions  and  iim  F (x}  » F(xj(  r Let  further  g{x)  be  everywhere 

n~t*oo  “ 

continuous  and  assume  that  to  every  e > 0 there  exists  an  A such 

that  f |fi(i)  j AFjjfi)  v e 0 Ih«n 

m &a 

AGO  ‘ ^*5 

lisa  f gU]  4F  (xj  « f six}  O.Flx)  . 
r* 

\ % * 

For  a proof  of  this  theorem  the  reader  is  referred  to  Cramer * 
®l>o  ©iSoo  p0  fM0 

It  is  easy  to  verify  that  the  conditions  of  the  HeXly-Bray 
theorem  are  satisfied  if 


lim  [F_(oo  j ~ F (“COJ  3 s?  Fioo  j — F(-o?  ) 
and  if  gixj  is  bounded 0 

$©  ©hall  first  show  that  ©v&?y  function  T$r(ej  given  by  (4o10j 
is  the  logarithm  of  the  0ofo  of  an  i0d0l0  For  this  it  will  be 
sufficient  to  show  that  ®vsry  fyis)  given  by  (4o10)  is  the  logarithm 
of  a 0ofo  sine©  it  is  obvious  that  with  ^(0)  also  idJLi  satisfies 


2q  see  this#  consider  first 


Uoi2) 


with  0 •;  £ <,  1 0 


I I®}  may  ho  written  as  the  limit  of  Riem&nn-StieXtjes  sums 

w 

(4C IS)  Sk"^[>.k{#  '"^-l)  ♦ isjxjj.]  v 

with 

xk 

Pk5*  > ~ 0'xk^ 3 • 

It  is  easy  to  v®r5.fy  that  ®13u  is  the  Ooto  of  the  dis- 
tribution of  & random  variable  which  equals  u with  probability 
o Mo  Hence  on  account  of  (4o7)  we  see  that 

l9Xy 

Xt-le  -U  * is£> 

X. 

is  logarithm  of  a ©ft£0  ana  so  consequently  is  0 Thus  X^Js)  is 


a limit  of  logarithms  of  ocf0sic  Also  I^ls)  is  obviously  continu 

oua  at  s*&G  0 By  u£t$%  $ continuity  theorem  Ig(  aj  is  thus  the 

logarithm  of  the  a0fc  of  a distribution  funotiona  By  the  $am® 
theorem  also 


% 


■ 
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IAb}  ~ Xim  Io ( 8) 


Q 


■J 


>e 


iax_  , lax 


AJSLj  1*^1  dG(x) 


la  the  logarithm  of  a 0of, 


But  dr(*) 

1 ❖<» 


ssiaa+jf  (eisx~l~  -iSS^)  d&Cx)  *t  IQ(s)  + isa -jp[G(0+)  - G(O-)  ] 

«=*  qo  x 


Is  obtained'  by  adding  to  2 (a)  * isa  the  logarithm  of  the  cof0 
of  a normal  &istrlbution0  Thus  SJtCg)  is  th©  logarithm  of  a o0f0 
Thus  the  equation  C4oX0)  is  sufficient  for  fyls)  to  be  the  logarithm 
of  a ©0f0  of  an  ipdoX0 

To  prove  also  the  neoeesity  of  (40XQ)  w©  need  sot&t&I  Xemmas0 
hamma  4„ln  a|xj  and  a in  (4o10)  are  uniquely  determined  by  fCa30 

»«fe  consider 

1 

UaM)  &Ib)  « X f><8> - ^2^b|i8^Ij  dh  * 

o 


-#■>0$  2 ♦00' 

j ei8XU-  ^ ac-{x5  * j*  ?ie*  &*<x) 

-do*  x *®oo 

where 

X £ 

(40l5)  $(s)  * /Cl™  &i§I)  At®.  aafy) 

-a>  • y J 
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4 

It  is  easy  to  verify  that 


(1-  SiSt) 


is  bounded  above 


and  below  by  positive  constants,,  Thus  |(x)  is  monotone  and  of 
bounded  variation,,  Xh©  Fourier  inversion  formula  determines  uniquely 
${x)  given  e(s)  and  thus  also 


x 2 -1 

U01&)  G(x)  ^ f &f){y) 

ur  y 

Finally  a is  determined  from  (doX0)o 

Lemma  4020  If  ^a(e)  determined  by  Gp(x)  and  by  an  converges  uni- 
formly in  every  finite  interval  to  a function  b(e)  continuous  at 

the  origin  than  Aim  £ (x)  =2.  G(x)  and  lim  a =s  a exist  and 

n-^oo  • n-*>o» 

bdsjss  ty(s)  Is  determined  by  a and  G(x)  by  equation  (4CXG)C 

i^roofg  F rcm  P0  Levy2 a continuity  theorem  It  follows  that 

b f s ) 

e lt  is  a 0ofc  hens©  every wh<sre  oontinuous0  Thus  also  b(s)0  it® 
logarithm,,  Therefore*  eal©)JP  defined  by  ^a{s)  by  m'sane  of  (4014) 

converges  to  a continuous  funotion0.  By  the  corollary  to  the  Continu- 
ity Theorem  it  then  follows  that  Lim  $n{x)  = ${x)  existec  More® 

n^oo  a 

over  |lx)  is  non-deor  easing  and  bounded,,  It  follows  from  the 
Helly-Bray  theorem  that 

Lim  G (xj  x.  Lim  fd,.  ?A£X.)  * &$niy) 

-go 

x i ? 

= fii-alsaf  A a$<y)  - s{x) 

-a?  * i+r 


8B 


since  the  intograni  Is  bounded*,  It  further  follows 0 also  from  the 
HeXly-Sray  th  sor-3mD  that 


*c© 

11m  I ( s ) — Urn  f (e 


isx 


lex 


■?) 


us-  »y*> 


*#•00 

A / isx 
=s  j ( e 


dG(x)  a*  1(b)  ^ 


'Finally  It  fallows  from  the  oonvergenoe  of  ^(s)  and  In(e)  that 
also  the  sequence  {aQ}  must  converse  and  thus  b(s)  — ^(s)0 

The  converse  of  lemma  402  follows  Immediately  from  the  Holly- 

Bray  theorem 0 

Lemma  £p30  Th«  0of 0 0(@)  of  an  i0d0I0  is  everywhere  different 

i 

from  zero0 

Conti  dor  {0(3}*^/^  We.  have  lire  ~ w(sl 

n-frco*”' 

where  w(s)  «s  1 for  0(b)^=O  and  w(g)  =5  0 for  0(s)  ~ Q 0 By 

f . • 

Levy's  theorem  w{g)  Is  a 0ofo  Since  w(s)  - 1 for  s and  w(s) 
is  continuous  being  a o0f0  we  must  have  w{s)  =.  l everywhere0 

\ 

Thus,  0 ( s J ^ 0 everywher a 9 

Lemma  4C4&  log  x » lim  n[x^n^  - 13,  x > O'  n Lemma  4-4  follows 
~~  n~^oa 

immediately  from  the  rule  of  4®  1' Hospital,. 

Lemma  4?50  If  0(b)  is  the  o0f0  of  an  i0d0l0  then  there  exists 

a aeguenoo  of  functions  ^|x)  given  by  (4o10)  gush  that 

log  0(g)-  lim 

a -*oo  a 
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Proof g W©  have  by  lemma  4C4 

log  0(6}  « 11m  n{[0(8)]ll/n)  -1}  * lim  ^ (s) 

n^co  n-^css  s 

uniformly  In  ©very  finite  interval  of  s since  0£s)=£o  with 

*00  x g 

&n  * a J £~*jpr  ^ Ga!xj  « nj  dFa{y)  0 

Hare  F (x)  is  the  &cfG  whose  0ofo  ia  [0(a)  3 a 

•Proo£  of  tha  necessity  of  {4  IQ) a By  lemma  403  0(s)^EsO  4?  hence 

log  0{s)  is  defiiied  everywhere  and  continuous  0 Moreover 

log  0{a}^  lim  &.(s)  uniformly  in  s with  Ir  (3)  given  by  (4o10)  0 

&-f»  CO  M » 

But  by  lemma  402  lim  ^n(a)s  4r(s)  where  ^(s)  1b  itself  deter- 

mined  by  (4.o10)o  Thus  theorem  401  Is  completely  proved0 

From  our  proof  of  the  sufficiency  ©f  equation  (4o10)  follows 
the  following  corollary  to  theorem  4010 

Corollary  t©  theorem  40X§  If  x is  distributed  according  to  an  I0do£0 
then  x s*  y * a wh@re  y is  normally  distributed  and  2 is  distri- 
buted &e  Is  the  limit  of  a sequence  of  finite  sums  of  independent 
random  variables  each  of  which  is  distributed  according  to  (4cs)  with 


V 


0 for  x < 0 

1 for  x > a 
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Theorem  4 aZa  Let  x.u  be  a differential  process  of  second  orderr 
than  ECxfc*T~xt^  251  1'm  VarCx^^-x^)  » t where  m and  g^  are 
constants  independent  of  t and  t0 

Froofg  he t ty^Cs)  be  the  logarithm  of  the  ocfc  of  x^^,  - x^  , From 
(4C9)  we  see  then  that  ^(s)  t^(s)  where  ^{e)  is  determined 

by  (4 ,.,10 } 0 Therefore  ty'(O)  = v^[(0)  and  ^(0)  =*  ^tx(O)  0 From 
this  and  from  0 ) - iE(xu<r-xt)  and  t^O)  = -VarU^-x^  5 
we  see  that  tho  iamaa  holds, 

. The  estimation  procedures  in  the  case  of  a process  given  by 
(404)  are  vary  $impla0  The  parameter  to  be  estimated  is  u f its 
maximum  likelihood  estimate  is  X-ffar0  the  number  of  shots  observed 
per  unit  of  time*  and  the  variance  of  this  estimate  is  ^/T  0 

If  the  process  is  given  by  1405)  then  the  increments  observed 

are  a sample  from  a population  with  the  distribution  ${r)  0 If 

» 

$lx)  is  given  in  parametric  form  then  the  proper  estimation  pro- 
cedures are  those  appropriate  for  estimating  the  parameters  of 
|ix)  from  the  observed  sample  of  increments*, 
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CHAPTER  5 


DIFFERENTIAL  PROCESSES  MODIFIED  BY  MECHANICAL  DEVICES 

X0  Filter  effect. 

In  registering  a stochastic  process  the  registering  device 
often  spread®  the  affect  over  a certain  period  of  time  in  such  a 
way  that  an  increase  occurring  at  time  t s*  0 will  produce  an  effect 
at  time  t and  the  observed  value 0 or  the  output  process,,  is  obtained 
from  the  superposition  of  all  the  effects  produced  from  increases 
that  occurred  in  preceding  time  periods 0 

If  we  let  y^  be  the  output  process  and  x^  the  input  process 


and  assume  that  the  effect  is  proportional  to  the  increase  we  then 


have 


C 


5 


yts  J f(t  -t)  d*f  c 

-CIO 


♦ C».  -»GQ> 

where  J"'  |f(t)|dt  and  J1  [f{t)J2  dt  and  the  integral  (oQl)  are 
- C£>  - CO 

assumed  to  exists 

taking  the  upper  limit  of  the  integral  ©qnal  to  infinity  instead 
of  zm?o  leaves  the  possibility  open  that  future  changes  will  in- 
fluence the  pre8.ezit0  If  the  present  is  not  affected  by  the 
future  j>  then  f(t)  will  be  0 for  negative  values  of  t0 

In  previous  work  we  have  often  talrsn  the  point  of  view  that 
the  process  start®  at  some  fixed  timi  T0  However  wo  can  also 

speak  of  the  conditional  distribution  "£  x given  x^  for  v < t 
B 

and  thus  f f(t-T)  &x  may  be  formed  for  every  A and  B and  we 
A*  T 


proceed  to  prove 


I 


. 
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Theorem  5010  Xf  x^  is  a differential  process  of  second  order  with 

-v-CO 

a weight  function  f{t)  then  the  integral  j fit-  %)  exists 

- oo 

loio^o 


In  the  following  v/e  use  a simplified  notation  by  writing 

H and  \ So*  H-H-x  o 

Proof  of  theorem  5eXS  tfe  have 


a p 

E[  j fit-t)  5xx3  s lim 

- ^ i 


f 


- lim  Eft^U-'cfjU.-x,  ,)2 
i 414  4 

• + 51f(t"Ti>f(t'Ti>{xi'xi-l>(x4-3C4-l)> 


whare  t,„x  < t|  <•  l 


Since  x^  is  a differential  prooess  of  second  order  we  have, 
writing  0| j for  the  covariance  of  (x^-  xi~l^  (x^-x^) 


0 if 

°S(Ti-  H~x'  if 


is  j 


E(xl-xl-l?  * aixi-  Vl!  • 


Thie  follows  from  theorem  4 020  • 


8? 


Thus 

B ■ B B 

(5*2)  E[  J tU  - T)dx,.32=  o2  ft8  (t  ~ T)fix  ♦ m2[ 

A A A 

t-A  ' t-B 

« cr2  j £2{t)<1x  * o2[  j*.  f (\)dx]6  c 
t-B  t-A 

Both  integrals  on  the  right  of  (5o25  converge  to  zero  if  A and  B 

converge  both  to  *<&>  or  to  - m 0 In  fact*  if  m^Q*,  only  the  con- 

GO 

vergenoe  of  J [f(t)]*dfc  need  be  assuznedgthus  by  lemma  l0a 

B 

l0i0m0  If  (t - Tjdx  exists. 

AH>  -00  J v 

■A 

We  denote  the  characteristic  function  of  the  increment 
~ a process  by  $^{s)  „ From  (409J  w@  see 

that 

(5,3)  fjU)  =r  exptt  log  ^(sjj 

We  compute  next  the  characteristic  function  of  y^  *>  that 

is  %C®3  - E(e'4syfc  ) 0 We  have 

|5,4j-  * E{exp[is  JfCt  -Tjtox]) 

-CO 

» E{axp[is  plim  Zf(t-rf  S(^1—  xui)l 
Aj~#*0  j 4 4*  A 


' 

. 


. 


■ 

; 


' 


' 

■ 


Stia  random  Tar  lab  Is-  in  the  exponent  at  the  rl$it  of  Cgi;/4).  converges 


In  probability  to 
i . tin  o t i on  o on  verges 


*3  j'  f(t-T)dx  rt  Hence  its  characteristic 
to  u*  Ja'j  c »<e  thus  have 


(5.5)  i)tU)  «U»  B{exp £is  Ztlt- xplxj-  *j-i) 


Since  the  summands  in  the  exponent  are  independent  random  variables 
we  have 


i*  (s$  - lira  TTE{exp[is-f(t 
%P®  j 


Th®  characteristic  function  of  xt  * therefore 

the  characteristic  function  of  £lt-> 


^Llsf(t  ~?||J  » exp  {A  j log  ^[sftt  «tJ 


(5*6)  , log  s|t(sj-  lim^Aj  log  fjtsftt-  t* 

4 


so  that 


* '00  * 00 

* J*  lag  (^[afU  - tJJAt  * J*  leg  ^[afUjJdt  0 
«.  w>  « oo 

The  oharaet'eristio  function  of  the  joint  distribution  of 

y,,  9 which  completely  determines  the  output  process  i@ 

*1  ‘ ' *n 

formed  In  an  analogous  manner  0 Me  have 
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fit  t ai»o»os3nJ  = E{exp  ! Js, yt  + a y )} 

''i  f o « o?  A n A *1  n j 


n 


♦ oo 

r% 


— E{exp  1 j tSjfftj.- t)+„..+8  -tj  3}dx  J 

-G2  z 


The  argument  employed  in  the  oase  nsl  shews  that 


<5o?J 


*lt-  o 0 t o o ©S> 

9 o o o 9 A 


♦ OP 


S j*  logi^LejfUj-  8nf{tn"  t^)d'c 


It  Is  saen  from  (50?)  that  ri+  f (biPoooSI8  ) and  thus  also 

»1  # OOOJ  A 11 

the  Joint  distribution  of  y^  9000?  ¥%  is  invariant  under  trans« 

1 n 

lations  in  timoc  lienee  th£  process  is  stationary 

If  the  distribution  of  x^  is  Gaussian  (a  differential  process 

which  is  Gaussian  is  a F0R0P0)  then  also  the  distribution  of  y* 

♦ oo 

is  Gaussian,,  The  variance  of  yv  is  given  by  ^ j*£f(t  }j2dt 

oS 

and  tae  covariance  function  is  given  by 


♦ oc 


♦ OB 


(5„8>  R(t  - t#  ) s E{[  f f(t-vj«SxT][  J*  f ( V - t )dx^ ] } 


a a 


-00 

£ r J Z 


j*  f(t~TT)f  It^T^dT  £S.  0*  j.  f(T)f{t#-  t+T^dT 


00 


114 ] A process  is  called  stationary  if  the  variables  x.  , o,0*x. 

tl  *n 

have  the  same  distribution  as  the  variables  x^  ^,0(8,xf  ^ 

1 n* 


0 


. . , ...  " ■ 


’ 

. 


'W 


* 


. 

■ : 


Umivn  e At  ( ' 9ass8  6 
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This  follows  immediately  by  writing  the  integrals  as  limits  of 
Kiemann-Stielt^es  sums  and  by  then  applying  theorem  5C1  and  lemma 
X040  Thus  the  resulting  process  is  a stationary®®  Gaussian  pro- 
cess with  covariance  function  (50G)0  If  we  put#  for  instance# 

for  t > 0 / 

0 tor  % <.  Q 


fit  J*- 


th©n  we  obtain  the  Q0U0P0  of  Chapter  Z with  Styles \|£ 

*P 


It  may  be  seen  from  C507)  and  (508J  that  a larg©  variety  of 
output  processes  may  be  obtained  from  .differential  processes,.  If 
the  differential  process  can  be  specified  in  parametric  form  and 
the  function  f{tj  la  also  known  at  least  in  parametric  form  then 
(5*7)  or  in  the  most  important  special  case  C S0B ) will  give  the 
output  process  in  parametric  form#  so  that  the  procedures  of  test- 
ing hypotheses  about  a finite  number  of  parameters  and  of  estimation 
in  the  parametric  ®ase  become  applicable  although  the  difficulties 
of  calculation  may  still  be  formidable,, 

la  case  nothing  is  known  about  either  fit j or  #f(s)  the  only 


way  known  at  present  by  which  some  inferences  can  be  obtained  is 
by  the  spectral  analysis  deoribed  in  Chapter  50 


If  | he  variables  x 


called/stationi 


A process  1 


y©  the  /lam®  dl^tributionlas  the/variable® 
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The  modifying  devloe  may  also  operate  in  such  a way  that 

the  modification  of  the  input  process  is  itself  dependent  on  pro- 

vlous  values  of  the  input  or  output  process0  A frequently  occur® 

ring  example  of  this  type  of  modification  is  provided  by  certain 

counter  devices*  which  count  random  events 0 Due  to  the  inertia  of  • 

the  counts  device  not  all  events  will  be  counted.  In  particular 

we  shall  consider  two  types  of  such  devices 0 

Type  10  After  an  event  has  been  registered  the  counter  remains 

locked  during  a certain  time  t . 

Type  After  an  event  has  happened  the  counter  remains 

locked  during  a certain  time  t c 

A general  and  comprehensive  treatment  of  probability  problems 

in  counter  devices  has  been  given  by  *»0  Feller  $Couraat  Anniversary 

volume*  1948$  ppc  105-115)  and  we  shall  here  follow  essentially 

Feller5 a representation  We  shall  assume  that  the  input  process 

is  a Poisson  process  described  by  (4C4)0* 

* 

Aet  T^  $ i > 1 be  the  time  interval  between  the  i-th  and 

the  (UX)st  registration^  the  time  from  the  beginning*  when 
the  counter  is  looked*  to  the  first  registration,,  Tbs  * i > 1 
are  independently  distributed  all  with  the  same  distribution,, 

*Ve  denote  the  time  up  to  the  (k+l}st  registration  by 

Let  I?  be  the  number  of  registrations  during  time  t0  #e  cl ear ~ 


ly  have 


• 

• 

* 

■ 

„ m 

‘ , ' ■ \ 


• 

' • • 


. 
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p.^t)  = Plfl-k)  ==  JflS^  < %)- <.  t) 

Let  Tk  s i>l  have  the  distribution  function  F , so  that  P(Tv  < t) 
sF(tj  0 vie  write  moreover  F C t ) for  the  distribution  function 

of  Ha  0 Let  Fk(t J-=  P(Sk  < t)  then 

(5C 10 5 pk(t)  « Fk_x(t)-  Fk(t)# 

Since  r and  since  and  5 are  Independent 

we  have 

t 

**n+1(t}  = JFn(t-z|  dFjsj  , 

© 

The  characteristic  function  of  the  random  variable  N 

is  thus  given  by 

qo  * . ■» 

(Sell)  P0(t5,^^  elBic[Fk_1(t)-Fk(t)] 

= P0Ct)  * e4sF0(t)  ♦ 

Thus  since  p@Ctj  ♦ P0tt)  « 1 p we  obtain 

♦*(*)  a 1 ♦ (e4e-l&  2sis\lt!  # 


($612j  • 
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Henoe 

(5013J 


{ S ^ X 0®  £ — T_ 

tt(s)=  * . = 

(®la- 1;  k*o 

= Fe(tl  + f 9is\^lt  ~x)dF(x 


“ P0t*)  * 9iS  jVsCBJdPU}. 
0 


In  type  1 as  well  as  in  type  Z counters  the  value  of  N is  hounded 

oo  Vv 

so  that  JL  converges*,  where  V,  » V-  (tj  is-the*k-th  mom@nt0 
k&o  * * 

Thus  w©  may  write 

* 9?  Yv  v 

^C©)  °s  X ^4i@j 
kso  *•* 


and  for  © < X also 


V8>=[|  § is  ♦ ♦...] 


-L 


isV« 


~ C Y-^  *o*  *$* 


*3  + . % 

^00*1# 


Henoe  the  constant  t©m  in  the  ^cpansion  of  ijr^(s)  beoomes  and 
the  aoeffiolent  of  (is)' 'becomes  tYg-Y^)/2  0 From  this  ami  (502*3j 

we  obtain  equations  for  Y^  and  Vg 


64 


& .14  K 


% 

VjCtj  « P0(t)  + JVj(t  -x)dF(x)  » 

0 


t 


Vg(t)  = 2Vx(t)  -F0(t)  ♦ jV2(t-  x}4F(x)  =;  EjH8) 

o 


»Va  begin  with  the  discussion  of  counters  of  type  i0  From 

at 

(4c.2j  we  see  that  F (t)  ^ 1 - e where  a > 0 ie  the  mean  number 

of  events  per  unit  of  time0 
Further 

Fit)  « X-  for  t > ^ 5 

F(t)  ^ 0 for  t < ? 

sine©  the  counter  remains  looked  during  the  time  ^ after  every 
registration  The  first  of  the  equations  Cs-X^i  then  becomes 


1~ 


•at 


for  t x 


{5^X5}  ^ - 1 ^ 

, * ajV^(t  - xje~a^s' 

1 * . 
tfe  compare  (s0l5)  with  the  more  general  equation 


for  t > % 


for  t <.  % 


Alt 


t 


t\  4- 


xio 


-ajx-?)  fQr  t > ta 


* 


at 


If  HCt)<l-6“6t  then  A(t)  < V,(t)  0 If  Hf.t)  > 1- 

100  | ' “=B  <Lr 

than  A(t)  > Y^lt)  0 This  Is  certainly  true  tor  0 < t < v and  can, 
easily  be  ^iosm  to  hold  for  t <,  (a  ♦ Xh  if  it  holds  for  t < n*  0 

y-w  '/  r^j 
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at 


rte  put  Alt)  7S  * o „0 


Than 


i * o for  fc  < x 

j !+&*£  ~ 


Bit 


\\  £i*S 

i8® 


1-  Lae” a ^ ^ for  t > it 

An  elementary  calculation  shows  that 


H(  % J < 1 - e~a*  if  OssO 
Bit)  > 1-  •' 


•*at  if  e«  sfj2 


irzTSsf 

Hence 

E 2 

A V«  £L 

C5.,isi  xdrf  < VtJ  i its?  * sn+afT 

It  la  also  possible  to  obtain  from  (5014)  an  exact  expression 
for  Y|(tj0  H owever* this  does  not  seem  to  be  of  great  interest 

since  C5ci$J  shows  that  approximates  V^C 1 1 very  closely  with 

a bounded  error  whieli  is  small  compared  to  Y^lt)  anless  a?  is  very 

large c The  exact  expression  for  V-^(t)  ie  moreover  very  Involved 

and  hard  to  evaluate 0 

H 

For  tha  Marianos  Bit)  of  H glvoa  by  B(t  J » Yg(t)  -£V^{t  jj  ^ 
Feller  found  the  asymptotic  expression 


(5,17) 


B(t ) = 


at 
(l+a**)' 


♦ o(t) 
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We  now  put 


00 


(5018)  •< 


tla)  s jVst  dF(t)  g fk(e) 
o 


« jVst  dFk(t 
0 


m 


fi(3)ta  jV1(t)e"8t  at 


we  have  by  Is, 

v^t)  = Zfcpjtj  * nkCPv.1it)-pk(t)j„  s Fjt) 

A ksl  ~ k=l  “ A K k«o  a J 


and  fey  induction^  using  the  well-known  multiplicative  property  of 

the  Laplace  transform*) 


ijjtaj  a ftt(8)[f(8)3k 
00  - 

wher © f I s ) s JV*0  * dF  { t ) 0 

o 


Thus 

Q©  , e©  co 

(5.1»)  *(#)*  II  jVSt  Ffc{t)dt  s £ | fe“St4FkJt)  « 

® 0 <0  “q>  a 

We  now  proceed  to  dismiss  counters  of  type  20 
teution  function'  F(t)  of  Tj^must  first  b©  obtained,, 


1 f0<8> 

¥ I~rl s j * 

The  distri- 
To  this  purpose 


we  shall  first  obtain  the  distribution  of  the  time  T during  which 
the  crnmtcr  is  lock@dc  The  probability  that  once  the  counter  is 


locked  exactly  v events  will' prolong  the  locked  time  T is  given 


by  4? P»  where  p «s  p"®*  8 qssl-  e~aT  $ since  p fey  i40E)  is  the  prefer- 
ability that  no  event  will  occur  during  time  t and  qir  the  probability 
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th-M  the  time  intervals  between  v successive  events  will  all  be  small- 

C 1 ^ 

er  than  t 0 T*st  now  T be  the  time  elapsed  between  the  (1-lJ^sb 
and  the  i-th  ev©ntD  The  total  looked  time  T0  provided  exactly  v 
events  prolong  the  looked  time0  is  thiin  given  by 

(5,20)  1 « f'1*  ♦ IC2J  I**1  ♦ T 0 

The  conditional  probability  'tf(t)  that  an  event  will  occur 
during  time  t provided  that  t 5 t is  then  given  by 


(50*V 

Thus 

*5.22 


Ufsjs  jV 

o 


&<*)«  «|  (l-e"at) 


st  AO(tl  - i.  JL,[1-  e"?a+8)T3 

q a<fs 


Ltt  now  v event e prolong  the  locked  time  and  write  (t 


000 


(v* 

❖ T < tjjvj  for  the  probability  that  the  locked  time 


will  be  at  most  t ♦ % « provided  v events  prolong  the  looked  time. 
From  f§022)  and  C50X8|  w©  see  that 


■ vt 

? 9”Ss  4W  ft)  * [ttisjf  0 

© 


{1}  ( v) 

Consider  now  v itself  as  a random  variable 0 Then  wCtjasP.lT  ♦oaoT.  •* 

^ O 

is  the  probability  that  the  looked  time  T is  at  most  t ♦ v c We 


thus  have 
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>"St  drtlt)  = pltaule)]**  z-jsxcfi  = Pi1"  af¥[1‘  «~(a+3|T3}”1 

„ ja»s^e"a<g  # 

~ 8+aa-ta+8^ 


Let  now  Gr(t)  s P(T  < t)  then  G(t)  — W(t-*c)  for  % > % 


and 


0 for  t < xn  Hence 


5o2s5  jV8t 
0 


«at 


cw 

•8*5  £^-8t 


je~ow  dW(t-T)  s Je“eb  dw(t 
^ o 


Ca  ♦ an-*8*8)* 

— ■ITT— H u, 

Ha4>a|^ 


a ❖ &e 


The  time  between  two  successive  registrations  is  composed 
of  the  resolving  time  T and  the  time  from  the  moment  when  the 
counter  is  free  to  the  next  event 0 The  distribution  of  the  latter 

eaft  j 

is  by  (402)  l~e  " and  has  the  Laplace  transform  a/ (a+s)  thus 


f(ai  - iV8t  AfCt  i - 

* - ^ J s ❖ a ekpi«^s 


whil®  fQ(e 


❖ s 


0 Substituting  this  into  (s019j  yields 


C 50^ ) 


fiU)  s aCs|a„6^^i 
8 (a«e) 


« 


' 


■ 


. ' 
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This  is  the  Laplace  transform  i$0±8)  of  the  function 


1~  © 


-at 


for  0 < % < n 


lo025} 


X-  ®’a£’  for  % > % 


Since  is  completely  determined  by  its  Laplace  transform* 

formula  i$0Z§))  gi^es  the  expected  number  of  registrations  during 
time  t in  a counter  of  type  £0  ^ 

A calculation  similar  to  the  on©  leading^! 5 D25)  shows  that 
the  Y&riEne©  B(tj  of  the  number  of  counted  ©Vents  ik  by 


C5o«6) 


B(»)  » Vg(t)-  [V1(t)3 


ae-aT(t-T)tl-  e“B's  ♦U»a*sf  e”2** 


GEA.PT  EH  6 

The  Fourier  Analysis  of  Stochastic  Processes,, 

A tJu^i  4 

A function  in  two  variables  is  called  raonotonoid  if 

f(t#V}  ss  gO^t'  I - ) where  g and  h are  two  functions  mono- 

tonia in  t and  t 7 in  the  same  sense0  rte  now  proveg 
Theorem  6q10  let  he  a stochastics  process  with  & monotonoid  and 

continuous  covariance  function  g^*  and  E{x$)  — * 0 then 

For  0 < t ^ T we  have  the  expansion 

ns^m 

(S01)  xt  = l0l0m0  2*  onexp[2irint/T] 

* Qs-rn 


wher  e 


T 


°a  ~ I jxt«xpt-2rtn*/T]  dt 

f ii  J This  limit  Is  uniform  in  0 < e < t <.  T-  s 

T T 

Uli)  %o ,=  4 j f au.  exp[.2m^SSl]dt  dt' 

f 0 0 


Ctv)  If  the  process  is  Gaussian#  then  any  finite  set 

of  0jx*®n  a^d  °nra  ^n  affe  Jointly  normally  dis- 
tributed*. 

To  simplify  the  proof  we  put  t s ^21  # y.r  s n , Th@n  f 

goes  from  G to  27T  as  t goes  from  G to  T and  have  to  prove  the 

formula 

{b-M  ^ ^ °in%  » °ns  iff  J^e'in%Ax  * 


We  thus  have  to  prove 


uniformly 


We  have 
$6„3J 


How 


7is~m 


Putt  ins 
(604) 


lim  Sty^J  ~y  ]£=0 

\ 

in  every  interval  £ < T < E7T-  e where 


r.M  _ in«- 

v x T Za  n ° 

" ni 


gff 

_(s)  i P „ 

;srj0  V 


*y5 

LzA 

-m 


’ >j  Ax' 


s e 


•ima  i_  ei(2m+l}a  e~ima  m e i(m+l)a 


1 «=  @ 


la 


1 08  e 


- e 


ima  _ eiCm*l)a_  ^Aima 


9 


*e 


iwiOTinnr  i wnw  v iWp 

£U~ 008  0) 


oos  ma  - Qoe 


gm+1 


s ' s t + h we  thus  have 


y 


(») 


E?r 


*.T-T 

I y 


T*h 


sin 

& 

sin  § 


£h 


and 


1 02 


.sq-t  ztr.T 


(6055 


e[4k)~  yT32 


L i 


r -r 


3in^ih  8iaS¥ik 

Vh,^k  — pk1  — T-V"  dh  dk 

* sin  sin  # 


35T  I 0' 


25T  J "''tpt+h  „ 

■ a.  h \n  ■g' 


, Sm^l: 

— % — ah  .♦  c 
h vx 


By  well-known  theorems  on  the  Dirichlet  integral l^aj  we  have 
uniformly  in  £ < *c  < 27T-&  is  > 0)p. 


(.6.6)  lim 

m~$>©  4Tf" 


1 L a^h» 


ain  8ln  Sgik 


C33»  V 

-IT 


v*k  . TT”’  . k 

sin  «g  sin  «g 


dh  dk 


ss  1 im  I o’—,  _ . v. 

sa~»oo^  i. 


ain 

- T ■ ■■  dh 

sin  4 


- a 


From  C 6C5|  and  id06)  it  follows  that 

(6.7)  lim  £ [y*m5  - y )2  , 0 or  l0i.m.  y^  a y 

uniformly  in  s < ? < 27T-  £ for  every  e > 0 0 


[l5]  For  the  double  Dirichlet  integral  see  Hobsons  "The  theory 
of  Emotions  of  a real  variable  and  the  theory  of  Fourier 
series"*  voXG  II*  pp0  7Qo-9c 


' : 


■ 

i - 


, ' r ? W'i  , J,  . > ' ■ '• 


■ 

, 

■ 


' 


, 


) 
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• v • is  completes  the  proof  of  the  first  two  statements  of  theorem 
, ii  - is  easily  obtained  by  an  elementary  computation  while 
7}  follows  from  the  representation  of  the  Fourier  coefficients 
«s  limits  of  Riamann  sums* 

% ft  ffrigon ome t ri o exTmng i on  of  t he  ?„  R»  ? „ 


as  an  example  we  shall  represent  the  F0RCP0  & trigono- 
,;.  IjIc  series  with  fandom  coefficient  s0  The  o ova? iano e f uiiv  t i o n • 
of  the  F0R0P0  Is  o minSt^t^J  [see  formula  C304)3&  this  is  a mono- 
mi©  and  non-dsereasing  function  of  t and  t*  so  that  theorem  60i 
i applicable 0 

.In  this  case  (that  1b  the  real  and  imaginary  part  of  on ) 


a 


; normally  distributed  with  mean  aero  and  we  have 


id0B)  ElF^,S  - ^ o min{ts»t'“  } =*  ')  s o'mlnfo*? 


I hue 


2Bp? 

Mon^m'i  ss  j"  ) expt- InT-loT^  3 d-t  d^! 

******  <f  © 


4 l 1» 


ar  gif 

^ p*  « 
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For  m^O  we  have 
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for  m 0 * 


For  0 wo  obtain  from  C6*9ag) 
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EC  o„q  | xg 
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If  we  write 


we  have 

- o„  * c „$>  *>  ss  i(o  - o j for  n > 0 * a 
& n « n*  n xx  -a?  o 

and  from  this  and  th©  formulae  (6f,10l3  - (6.0905)  we  find 


E(&o)  a OT/3 

E(a0anj  a -cT/2TT*n 

Eia^a^j  s 0 for  is^n,  n^O,  n^O 

E(a2)  — eT/2n^n2 

iSa10j  < 

] W “ 0 for  n£  0 

E(a  b^)  s ~cT/27Tm 

= 3oT/2r3n8 

= oT/rr ran  for  ra^fc  n* 
n in/  * < ' 


rfe  shall  now  estimate  eTx  ~ £ jm H **  for  a fixed  a wher® 


, ax  a 

xim*  -s  a ♦ ,S  a 003  arfit  ♦ 21  b sin  Efi^i 
% ® n«l  n (F  Ra  1 11  T 


oo 


n*«H4i 


tfe  havOg  using  (8CX0) 
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E[xt-  x^a?]2s  ZL  E(a2)ooa2  2lfS£  « 21  Ei'e^ela**  2ff£S 
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00  00 

2 i„.  S{Vbk)Bln  2jr~i  sin  2u 
05  05  . antni2 
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= sh  £ V jf -ia 

fl*  2 ^ n4  ! L-* 

L n«mW  ~ 


si 


Expanding  the  function  7T- a Into  a Courier  series  we  get 


nr-o  » zE  SiS_SS, 
n*l  a y 


heno  e 


055 
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sin^  na 
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si 

, IlTT-  s'j  - S SiSLM  m f ( a), 
a n*i  n 


Differentiating  this  equation  we  have 
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* i ain(mv|ja 

« ri  S=>  <— ■BBaegs  -^xg^*xrr’“.fc> 


f ^ (a)  - - i - Haas  n * ~ „ _ 4 

2 ^ 2 8lnf  ^ 


and  therefor® 


os 

T sin  na 
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Hence’ 


* A ,2. 

2 . 1 f 8tn(cH-2)t  dt  _ tt  _ j 8ln(2a-»l)Y  dv 

2 2 1 sin  | “2  - 


E£zt-  4“>]2  s S!fi  f -i-  , ar^r-sin^IjY  1 

" ’ 1*1*  fan*  ‘-s  J • S“  " ' j 


Thus  far  values  of  t not  t.vo  close  to  0 or  T ff  xi  J is  a good 
approximation  to  x^  0 

Another  and  perhaps  are  useful  formula  may  he  obtained  by 
deriving  the  following  expansion 


t T T 
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In  this  expansion  the  and  b^  are  independently  and  normally 
distributed  variables  with  mean  zero  and  variances  * the 

aAs 

an  and  bn  ar©  also  independent  of  e Th©  right  side  converge^ 

moreover^ uniformly  in  the  mean  to  the  left  side0  The  proof  can 
be  obtained  by  first  applying  theorem  s0l  to  the  stochastic  pro- 
cess ac^-^Xij  and  determining  the  Fourier  coefficients  end  their 

variances 0 It  is  then  seen  that  the  Fourier  expansion  thus  obtained 


oo 


converges  also  l0i0ra0  for  tsl  and  thus  - Z &&  o proof 

w'  n«X 


Is  rather  laborious  but  elementary  and  is  therefore  omitted, 
Thus  writing  x^j  33  &a  w@  have 


m 

<6oU)  *t  “ ao  t * & {»atoos^-13  ♦ bn 


where 


02  * 
ao 


0S  • of  * «§ 


cT 


n 


2lPn‘ 


. Vj'Vj"0  1#J  ’ Vi3  0 


Except  for  the  constant  term*  - £a  . this  Is  essentially  the 

j n»i n-> 


expansion  discovered  by  PaXey  and  Wiener,; 


[16] 


[163  Fourier  transforms  in  the  complex  domain^  p0  14? 0 
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S 0 Stationary  processes. 

f©  now  return  to  the  general  theory  and  consider  stationary 

processes 0 


exist  a o We  then  have  cr^.*  =5  where  R(xJ  is  an  ©¥@n  func- 

tion Of  T o 

We  she.ll  also  consider  a slightly  more  general  class  of 
processes*  called  guasi stationary  processes 0 A process  x^  is 

aid  to  he  quasiatationary  if  E[x^]  is  independent  of  t and  if  its 

covariance  function  exists  and  is  given  hy  cr^t*  zz  R{t~  £'}  where 

R|x)  is  an  even  function  of  t 4 

we  assume  now  that  R(t)  is  continuous  at  the  point  x^O 
ad  show  that  R(tJ  is  then  continuous  everywhere 0 If  R|tJ  is  con- 


tinuous at  tsO  then  we  have 


lim  E(xt+T-xt)2  s 0 „ 


From  the 


definition  of  Rlxj  w©  see  that 


h~>© 


lim  iR{%+h)  -R(x)J  = HmE[’(x_  . -x  }x  0 
.-®*o  h^o  c,u  y 
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w le  next  introduce  the  following  definition 3 A function  f(t) 
is  said  to  be  positive  definite  if 

(a)  f(t)  is  continuous  end  bounded  on  the  real  axis* 

(b)  f(t)  is  Bicrmitianj,  that  is  £(~t)  s f( t fg 

(c)  for  any  positive  integer  m and  any  real  numbers 
\9  zz»ooob  zm  and  any  complex  numbers  ugS? 

0009  ^ we  have 


From  the  preceding  it  is  clear  that  H(t)  satisfies  conditions 
{a}  and  (b)  since  H(t)  is  real  and  ev©n0  ue  have  only  to  prove 
that 


m m 


h-X  k*l 


,?t  A f^2h"  zk^uhuk  ~ 0 0 


m m 


3 - Z Z B(th-  vv\  - 0 0 


We  have 
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function^ 
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According  to  a theorem  of  S0  Boolmor^7^  every  positive 
definite  function  fit}  may  he  represented  in  the  form 

* oo  ^ 

fit)  a J eita  dV(a) 

~0D 


where  Via)  ie  a hounded  non^decreasing  funotiona 
Thus  w©  have 


I 60l5) 


R(t) 
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= S e1*® 
~oo 


dgl  <o) 


where  gl©}  is  a bounded  and  non-decreasing  function0  we  may  take 
g(-oo)  - 0 D Then  gloo)  s&  R##and  glct}/R(d)  could  therefor©  be 


defined  as  a distribution  funotlon0  It  will  however  simplify  our 
formulae  If  we  determine  gla)  so  that 


«(a)=  &Latl  ± .sUdl.  . 
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Binoe  Hit}  ss  R{-tj.we  have 
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tl?3  Sc  Bochner9  Yorlesungen  ubar  Fouriersch®  Int©grales  p0  7Q& 
Sats  23  0 


Ill 


end  ainoe  the  function  gUo)  is  unions  if  g(  = ®!  s 0 
g(o5  s £L&l~±JSk$zl9  w©  must  have  g{»5  s g{oo  } « g(~$) 

H 

end.  fa?  osO,  g(co)  = 2g{0)  and  g{®)  * g(O)  = g{0)  » g(=a). 
It  Is  further  well  known  that 

, -2  eitffl-, 

g(K)-g(0)  = inmii5.JR(t)_=i  dt  0 
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//e  may  also  writ© 


R( t i ss 


♦®eit®A„-n® 
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OO  00 
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where 
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(6oi6)  < 


fC®j  - gl©)-gU©)ss 
Fjce  ) S g(co  } S R(0) 


FiOj  s 0 , 


Further 


?{») 


g(oo)  - sI-cd)  = 11m 
T 


so  that 


Fla)  5 | JrU)  at  c 

G 


It  may  also  be  remarked  that  to  every  positive  definite 
function  R(t)  we  may  construct  a Gaussian  process  with  R(^)  as 
covariance  function*  i'his  can  he  done  by  defining  the  distribution 

of  xt  ? 9 0 08  n to  be  a multivariate  Gaussian  distribution  with 

1 n 

covariance  matrix  JjRtt^-tj)  |{  0 Since  R{t)  is  positive  de- 
finite auoh  a distribution  always  exist s0  It  is  then  easy  to 
verify  that  the  family  of  distribution  functions  so  defined  satis- 
fies  the  consistency  conditions  of  chapter  Xc  Combining  this  with 
the  result  of  Boohner  we  obtain 

Theorem  6QS0  The  function  R(t)  is  the  covariance  function  of  a 
quasistationary  process  If  and  only  if  it  is  the  Fourier  transform 
of  a bounded  non-decreasing  function* 

4 0 The  mean  ergodio  theorem, 

V/e  shall  conclude  this  chapter  with  a proof  of  the  mean 
orgodic  theorem0 

Theorem  5050(Mean  ergodio  theorem  \ g Bet  be  a quasi  stationary 

process  with  continuous  covariance  function  R(t)  and  mean  value  zero 


[16]  This  theorem  is  due  to  J0  vc  Neumann  g Proo  0 Hat,  Aoad0  Soie9 
vol.  1811932),  pp.  70-8S. 
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Then 
{ 6el7 ) 


1 o 1 O ^ ft 

T ~*$*G0 


fit 
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dt  ~ 


ax 


where  is  a random  variable  with  variance  g(X+)-g(X-J  and  mean 

z@ro  and  where  E(a^a  ) «•  0 for  X4* -p.  ft  The  function  g(Xj  1b 

defined  hy  ld0l5}0 

w©  first  prove  the  following  lemma 

Lemma  60lg  For  any  0 < * < % # T > 1 aad  for  ©V9ry  £ 

dx-  [ga^j-g(\-|)] 
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♦ ■g'L  g(^«+  £ J 


4 


£)3*  «Rr[g(ao')-  £(~gd  }] 


Proofs  *ve  put 


T 

-»  f &{t-T 

1 ! e 

ai’ 


^ 3!  ^GO 

d-v  ?»  i P fexp[iX(t-T}*lo)|  t«Tj  J&gCaj)  d*s 

* j J 

Q-CO 


:s  J1  * J2  * J3  + h * Sz  * *T3 


where 


i 


114 


x r iMt-^3  | 

” if*  eft 


1 Is  $ ,18 

o 


*.  i f 

dgu-'dir  | Jjss  |>  j 


“t)j 


a""4,05'  '"^dg!®)^ 


X f*iXSt”'E}  f _i©(t“?) 

fj8  j " 


dg(»)&Tj  Jsa  f | ®~*®it*T^dglo)d,f 

<&> 

£ 
r 


: ~ i ! oie)U’';) 

T ^ 

Q 


| Jr>  ^ i 

I 3 2 


T 

9 


i 

mxsS 

T 


ioC t^v) 


$ 


These  Integrals  converge  absolutely,,  Henoe  we  may  Interchange  the 
order  of  integration  whenever  necessary,,  In  this  manner  w#  obtain 
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a^|  ^ ji 

, .T 


i(<D*X)t 


so  that 
(6*18) 
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using  this  inequality  we  obtain  from 
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jexp[iOv*s)Mt-T)]dT  igiio) 
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|6C19)  |I2]  < l[g(x+e  - g(X+  +g(-X+e  - g(-X+  ] 
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and  similarly 


(6cX9a)  jJgJ  < ^~[glX+e)  - gDw4-}  -s-gi-X^s)  - g(~X*)  ] 


Since 


|[g(X«£)-g(x-£)]  = A J JVgU)  fiT 

tj/4  o 


we  have 0 using 
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Sine® 


lift 


X > I and  sine©  g(x)  is  non- decreasing  it  is  seen  easily  that 


( 6 


20}  j Xg  - - s(X-£)J  | - gfX-©}  ] 


Similarly  we  obtain 


(6o80a) 


e<T«» 

”2®' 


t) 


X*c 
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Lemma  60X  then  follows  easily  from  C60ls),  (60X8a}*  i60I9)£  (6oX9a)0 

(6o20j»  (6o20&)o 

Corollary  1 to  lemma  6 010 


T 

(602l)  Ilia  U f e^lW’H(t-T}d-E  * gCX*)-g(X”)  * 
X ~*»00  1 % 


Corollary  2 to  lemma  6«lft 

* x*t*  »mmrm  I r>  w ■**■»*■  Jrea^>a»«g^«»r*  Tifci^ia^KJW  UJiTTMBwMiT^er^r  lfcMiaW 


(6C22) 


T»  rr 


gfo* ) - g(X- ) 0 


'Proofs  may  always  write  the  double  integral  so  that  Tf  > X 
so  that  lemma  601  is  applicable  and  corollary  2 followB  easily 
sine a e is  arbitrary0 
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In  tha  proof  of  the  mean  ©rgodic  theorem  w®  shall  operate 

/ 

with  complex  random  va3:*i&bXes0  If  zs^ij  is  a complex  random 
variable  with  mean  zero  we  shall  define 


( 60SS) 


s Et  zz)  S o|  ♦ 0* 


where  zgx-  ij  is  the  complex  eonjy^gate  to  zc  A sequence 
{sa}  s {xn4-i'yn}  of  complex  random  variables  converges  if  both 

{x^}  and  {yn}  converge,.  From  lemma  lc8  it  follows  that  {za} 
converges  l0icm0  if  and  only  if  B[tzn- zm)  zm)  ] is  arbitrarily 
email  for  sufficiently  large  n*  a 0 
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To  show  that  X^a  A at  converges  in  the  mean  we 
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consider 
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All  thro©  integrals  converge  to  the  same  limit  by  (6022)0  Thus 
l0i0m0  X « a*  exists 0 Moreover * by  lemma  X07  and  (6022) 
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For  we  further  have 
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The  second  integral  converges  by  corollary  1 of  lemma. 601  to 
g(p.*)  - g(p,-)  uniformly  in  t 0 Thus' 

r J 
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T«^co  ^ 0 


where  lim  ji(T)  ss.  0 0 It  easily  follows  that  E(au  a } ss  0 0 
’ T A A 

Theorem  8~4C  Let  be  any  quasistationary.  process  with 

function  R{^)  and  let  gC©)  be  defined  by  {©0l5)0  Further  let 
Xg^ooo  ba  the  discontinuities  of  g(s>)  and 
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Functions  and  Activities 

The  National  Bureau  of  Standards  is  the  principal  agency  of 
the  Federal  Government  for  fundamental  and  applied  research  in  phys- 
ics, mathematics,  chemistry,  and  engineering.  Its  activities  range 
from  the  determination  of  physical  constants  and  properties  of  ma- 
terials, the  development  and  maintenance  of  the  national  standards 
of  measurement  in  the  physical  sciences,  and  the  development  of 
methods  and  instruments  of  measurement,  to  the  development  of  special 
devices  for  the  military  and  civilian  agencies  of  the  Government. 
The  work  includes  basic  and  applied  research,  development,  engi- 
neering, instrumentation,  testing,  evaluation,  calibration  services, 
and  various  scientific  and  technical  advisory  services.  A major 
portion  of  the  NBS  work  is  performed  for  other  government  agencies, 
particularly  the  Department  of  Defense  and  the  Atomic  Energy  Com- 
mission. The  functions  of  the  National  Bureau  of  Standards  are  set 
forth  in  the  Act  of  Congress,  March  3,  1901,  as  amended  by  Congress 
in  Public  Law  619,  1950.  The  scope  of  activities  is  suggested  in 
the  listing  of  divisions  and  sections  on  the  inside  of  the  front 
cover. 

Reports  and  Publications 

The  results  of  the  Bureau's  work  take  the  form  of  either  actual 
equipment  and  devices  or  published  papers  and  reports.  Reports  are 
issued  to  the  sponsoring  agency  of  a particular  project  or  program. 
Published  papers  appear  either  in  the  Bureau's  own  series  of  publi- 
cations or  in  the  journals  of  professional  and  scientific  societies. 
The  Bureau  itself  publishes  three  monthly  periodicals,  available 
from  the  Government  Printing  Of f ice : the  Journal  of  Research,  which 
presents  complete  papers  reporting  technical  investigations;  the 
Technical  News  Bulletin,  which  presents  summary  and  preliminary  re- 
ports on  work  in  progress;  and  Basic  Radio  Propagation  Predictions, 
which  provides  data  for  determining  the  best  frequencies  to  use  for 
radio  communications  throughout  the  world.  There  are  also  five 
series  of i nonperiodical  publications:  the  Applied  Mathematics  Se- 
ries, Circulars,  Handbooks,  Building  Materials  and  Structures  Re- 
ports, and  Miscellaneous  Publications. 

Information  on  the  Bureau's  publications  can  be  found  in 
NBS  Circular  460,  Publications  of  the  National  Bureau  of  Standards 
($1.00).  Information  on  calibration  services  and  fees  can  be  found 
in  NBS  Circular  483,  Testing  by  the  National  Bureau  of  Standards 
(25  cents).  Both  are  available  from  the  Government  Printing  Of fice . 
Inquiries  regarding  the  Bureau’s  reports  and  publications  should  be 
addressed  to  the  Office  of  Scientific  Publications „ National  Bureau 
of  Standards,  Washington  25,  D.  C. 


